COMPUTATIONAL COSTS

FreeEval: A Modular Framework for Trustworthy and Efficient Evaluation of Large Language Models (2024.emnlp-demo)

Challenge:	Inference of LLMs incurs high computational costs, memory access overhead, and memory usage, leading to inefficiencies in terms of latency, throughput, power consumption, and storage.
Approach:	This tutorial introduces the basics of efficient inference for LLMs and explains how to diagnose efficiency bottlenecks for a given workload on specific hardware.
Outcome:	The tutorial introduces the basic concepts of modern LLMs, software and hardware.

Copied to clipboard

Zhuohao Yu, Chang Gao, Wenjin Yao, Yidong Wang, Zhengran Zeng, Wei Ye, Jindong Wang, Yue Zhang, Shikun Zhang

Compact Multimodal Language Models as Robust OCR Alternatives for Noisy Textual Clinical Reports (2026.eacl-industry)

Challenge:	Large language models (LLMs) have revolutionized natural language processing with impressive performance across various tasks.
Approach:	They propose a framework for automated evaluations of large language models . they open-source their code at https://github.com/WisdomShell/FreeEval .
Outcome:	The framework is open-source and can be used to develop and validate new evaluation methods.

Copied to clipboard

Nikita Neveditsin, Pawan Lingras, Salil Patil, Swarup Patil, Vijay Kumar Mago

Extracting Text Representations for Terms and Phrases in Technical Domains (2023.acl-industry)

Challenge:	Conventional OCR systems perform poorly under noisy, real-world conditions . compact multimodal models outperform classical and neural OCR pipelines .
Approach:	They evaluate compact multimodal language models for transcribing noisy medical documents . they compare eight different models to find the best transcription accuracy and noise sensitivity .
Outcome:	The proposed models outperform classical and neural OCR pipelines in transcription accuracy, noise sensitivity, numeric accuracy and computational efficiency.

Copied to clipboard

Francesco Fusco, Diego Antognini

Grouped Sequency-arranged Rotation: Optimizing Rotation Transformation for Quantization for Free (2025.acl-srw)

Challenge:	Large pre-trained language models are extensively used in modern NLP systems.
Approach:	They propose an unsupervised approach to encoding using character-based models and pre-trained sentence encoders to reconstruct large pre-trained embedding matrices.
Outcome:	The proposed approach matches the quality of sentence encoders in technical domains and is 5 times smaller and up to 10 times faster on high-end GPUs.

Copied to clipboard

Euntae Choi, Sumin Song, Woosang Lim, Sungjoo Yoo

Does Simultaneous Speech Translation need Simultaneous Models? (2022.findings-emnlp)

Challenge:	Existing rotation-based methods struggle at low bit-widths, such as 2-bit weight quantization.
Approach:	They propose a training-free approach to construct an improved rotation matrix . they leverage the Walsh-Hadamard transform with sequency ordering to reduce quantization error .
Outcome:	The proposed method demonstrates robust performance on reasoning tasks and high Perplexity (PPL) score on WikiText-2.

Copied to clipboard

Sara Papi, Marco Gaido, Matteo Negri, Marco Turchi

Beyond Unified Models: A Service-Oriented Approach to Low Latency, Context Aware Phonemization for Real Time TTS (2026.eacl-industry)

Challenge:	Simultaneous speech translation (SimulST) systems strive for high output quality but also low latency.
Approach:	They propose to train SimulST offline without additional training or adaptation . they also show offline training achieves similar or better quality compared to offline training .
Outcome:	The proposed model can serve both offline and simultaneous applications without additional training or adaptation.

Copied to clipboard

Mahta Fetrat Qharabagh, Donya Navabi, Zahra Dehghanian, Morteza Abolghasemi, Hamid R. Rabiee

MediSwift: Efficient Sparse Pre-trained Biomedical Language Models (2024.findings-acl)

Challenge:	Lightweight, real-time text-to-speech systems often rely on lightweight phonemizers . a new framework aims to bridge the trade-off between phonemization quality and inference speed .
Approach:	They propose lightweight strategies for context-aware phonemization and a service-oriented TTS architecture that executes these modules as independent services.
Outcome:	The proposed system improves pronunciation soundness and linguistic accuracy while maintaining real-time responsiveness.

Copied to clipboard

Vithursan Thangarasa, Mahmoud Salem, Shreyas Saxena, Chen-Yu Leong, Joel Hestness, Sean Lie

Japanese Predicate Conjugation for Neural Machine Translation (N18-4)

Challenge:	Large language models are typically trained on general source data forvarious domains, but domain-specific pre-training is expensive and requires computational costs.
Approach:	They propose a suite of biomedicalLMs that leverage sparse pre-training on domain-specific biomedically text data.
Outcome:	The proposed model outperforms existing LLMs on biomedical tasks by 22.5x .

Copied to clipboard

Michiki Kurosawa, Yukio Matsumura, Hayahide Yamagishi, Mamoru Komachi

Graph-based Filtering of Out-of-Vocabulary Words for Encoder-Decoder Models (P18-3)

Challenge:	Neural machine translation (NMT) has a drawback in that it can generate only high-frequency words owing to the computational costs of the softmax function in the output layer.
Approach:	They propose two methods to generate low-frequency words and deal with unknown words using Japanese predicate conjugation information without discarding linguistic information.
Outcome:	The proposed methods can generate low-frequency words and deal with unknown words.

Copied to clipboard

Satoru Katsumata, Yukio Matsumura, Hayahide Yamagishi, Mamoru Komachi

Graph-based Multilingual Product Retrieval in E-Commerce Search (2021.naacl-industry)

Challenge:	Encoder-decoder models employ words that are frequently used in the training corpus but may still include noisy words.
Approach:	They propose a method for selecting more suitable words for learning encoders by utilizing co-occurrence information.
Outcome:	The proposed method outperforms the baseline method in Japanese-to-English translation and grammatical error correction tasks with an F-measure of 1.48 points higher.

Copied to clipboard

Hanqing Lu, Youna Hu, Tong Zhao, Tony Wu, Yiwei Song, Bing Yin

Long-range Sequence Modeling with Predictable Sparse Attention (2022.acl-long)

Challenge:	Modern e-commerce search systems require product retrieval under multilingual scenarios.
Approach:	They propose a universal multilingual retrieval system that captures interactions between search queries and items in e-commerce search.
Outcome:	The proposed system outperforms state-of-the-art retrieval models on five countries and has been deployed in production for multiple countries.

Copied to clipboard

Yimeng Zhuang, Jing Zhang, Mei Tu

Mixture of Heterogeneous Grouped Experts for Language Modeling (2026.acl-industry)

Challenge:	Existing approaches to capture global context dependencies in sequence modeling suffer from quadratic complexity in time and memory usage.
Approach:	They propose an efficient Transformer architecture for fast long-range sequence modeling with a sparse attention matrix and a hidden state cross module.
Outcome:	The proposed architecture outperforms the standard multi-head attention and its variants in various long-sequence tasks with low computational costs.

Copied to clipboard

Zhicheng Ma, Xiang Liu, Zhaoxiang Liu, Ning Wang, Yi Shen, Kai Wang, Shuming Shi, Shiguo Lian

Towards efficient self-supervised representation learning in speech processing (2024.findings-eacl)

Challenge:	Large Language Models (LLMs) based on Mixture-of-Experts (MoE) enforce uniform expert sizes, creating a rigidity that fails to align computational costs with varying token-level complexity.
Approach:	They propose a mixture of heterogeneous grouped experts (MoHGE) that allows for flexible, resource-aware expert combinations.
Outcome:	The proposed model matches the performance of existing Mixture-of-Experts architectures while maintaining balanced GPU utilization.

Copied to clipboard

Luis Lugo, Valentin Vielzeuf

CuriousLLM: Elevating Multi-Document Question Answering with LLM-Enhanced Knowledge Graph Reasoning (2025.naacl-industry)

Challenge:	Existing models require several GPUs for days to pretrain, generating environmental concerns because of their high energy consumption.
Approach:	They propose an efficient self-supervised model that uses a single GPU during 24 to 48 hours of pretraining to address high computational costs.
Outcome:	The proposed model represents two orders of magnitude better than existing models.

Copied to clipboard

Zukang Yang, Zixuan Zhu, Jennifer Zhu

A Benchmark and Evaluation of Automated Language of Study Extraction from Computational Linguistics Publications (2026.eacl-srw)

Challenge:	Large Language Models (LLMs) have achieved significant success in open-domain question answering, however, they continue to face challenges such as knowledge cutoffs and hallucinations.
Approach:	They propose a new mechanism that integrates a curiosity-driven reasoning mechanism into an LLM agent to generate relevant follow-up questions.
Outcome:	The proposed enhancement integrates a curiosity-driven reasoning mechanism into an LLM agent, enabling it to generate relevant follow-up questions, thereby guiding the information retrieval process more efficiently.

Copied to clipboard

Henry Gagnier, Ashwin Kirubakaran

Learning to Generate Word- and Phrase-Embeddings for Efficient Phrase-Based Neural Machine Translation (D19-56)

Challenge:	Language of study extraction is an aspect of computational linguistics papers that is useful for analyses of trends and diversity in computational linguists.
Approach:	They propose to benchmark and evaluate automated language of study extraction from computational linguistics papers.
Outcome:	The proposed language extraction benchmarks show that they can extract languages from papers with accuracy without high computational costs.

Copied to clipboard

Chan Young Park, Yulia Tsvetkov

Semantic Exploration with Adaptive Gating for Efficient Problem Solving with Language Models (2025.acl-long)

Challenge:	Neural machine translation (NMT) often fails in one-to-many translation, e.g., in the translation of multi-word expressions, compounds, and collocations.
Approach:	They propose a phrase-based NMT model that generates embeddings of words or phrases.
Outcome:	The proposed model performs on par with state-of-the-art phrase-based NMT.

Copied to clipboard

Sungjae Lee, Hyejin Park, Jaechang Kim, Jungseul Ok

Less is More for Long Document Summary Evaluation by LLMs (2024.eacl-short)

Challenge:	Existing methods for multi-step reasoning suffer from inefficiency and redundancy . existing methods neglect the diversity of task difficulties leading to extensive searches even for easy tasks .
Approach:	They propose a method that explores reasoning paths with a gating mechanism that decides whether to conduct a tree search based on the confidence level of answers from a previous simple reasoning method.
Outcome:	The proposed method significantly improves accuracy by 4.3% on average while requiring only 31% of computational costs.

Copied to clipboard

Yunshu Wu, Hayate Iso, Pouya Pezeshkpour, Nikita Bhutani, Estevam Hruschka

Challenge:	Large Language Models (LLMs) have shown promising performance in summary evaluation tasks, yet they face challenges such as high computational cost and the Lost-in-the-middle problem where important information in the middle of long documents is often overlooked.
Approach:	They propose a novel method which extracts key sentences from a long source document and then evaluates the summary by prompting LLMs.
Outcome:	The proposed method significantly reduces evaluation costs and exhibits a higher correlation with human evaluations.

Towards Federated Low-Rank Adaptation of Language Models with Rank Heterogeneity (2025.naacl-short)

Copied to clipboard

Yuji Byun, Jaeho Lee

Enhancing Software Requirements Engineering with Language Models and Prompting Techniques: Insights from the Current Research and Future Directions (2025.acl-srw)

Challenge:	Low-rank adaptation (LoRA) is an efficient alternative to full-weight adaptation in federated fine-tuning of language models, significantly reducing computational costs.
Approach:	They propose a low-rank adaptation method that freezes original weights and trains only the update parametrized as a product of two low-ranked matrices.
Outcome:	The proposed method accelerates convergence and enhances the global model’s predictive performance.

Copied to clipboard

Moemen Ebrahim, Shawkat Guirguis, Christine Basta

Federated Retrieval Augmented Generation for Multi-Product Question Answering (2025.coling-industry)

Challenge:	Large Language Models (LLMs) offer transformative potential for Software Requirements Engineering (SRE), but they face challenges such as domain ignorance, hallucinations, and high computational costs.
Approach:	They propose a conceptual framework that integrates Small Language Models and Knowledge-Augmented LMs with LangChain to address these limitations systematically.
Outcome:	The proposed framework addresses six technical challenges and two research gaps through a systematic review of LLM applications in software requirements engineering.

Copied to clipboard

Parshin Shojaee, Sai Sree Harsha, Dan Luo, Akash Maharaj, Tong Yu, Yunyao Li

A Semantic Uncertainty Sampling Strategy for Back-Translation in Low-Resources Neural Machine Translation (2025.acl-srw)

Challenge:	Existing multi-domain RAG-QA approaches query all domains indiscriminately or rely on rigid resource selection.
Approach:	They propose a multi-product knowledge-augmented QA framework with probabilistic federated search across domains and relevant knowledge.
Outcome:	The proposed framework improves multi-product knowledge-augmented QA performance on Adobe products.

Copied to clipboard

Yepai Jia, Yatu Ji, Xiang Xue, Shilei@imufe.edu.cn Shilei@imufe.edu.cn, Qing-Dao-Er-Ji Ren, Nier Wu, Na Liu, Chen Zhao, Fu Liu

Skill Discovery for Software Scripting Automation via Offline Simulations with LLMs (2026.findings-eacl)

Challenge:	Back-translation methods rely on large-scale parallel corpora to enhance performance, but ignore the semantic quality of monolingual data.
Approach:	They propose a method which prioritizes sentences with higher semantic uncertainty as training samples by computationally evaluating the complexity of unannotated monolingual data.
Outcome:	The proposed method improves translation accuracy and fluency by +1.7 on all three translation tasks.

Copied to clipboard

Paiheng Xu, Gang Wu, Xiang Chen, Tong Yu, Chang Xiao, Franck Dernoncourt, Tianyi Zhou, Wei Ai, Viswanathan Swaminathan

QUPID: Quantified Understanding for Enhanced Performance, Insights, and Decisions in Korean Search Engines (2025.acl-industry)

Challenge:	Large Language Models (LLMs) can generate code from natural language queries, but runtime code generation is limited due to unverified code, security risks, longer response times, and higher computational costs.
Approach:	They propose an offline simulation framework to curate a software-specific skillset by exploiting large language models and publicly available scripting guides.
Outcome:	The proposed framework significantly improves automation success rates, reduces response time, and saves runtime token costs compared to traditional runtime code generation.

Copied to clipboard

Ohjoon Kwon, Changsu Lee, Jihye Back, Lim Sun Suk, Inho Kang, Donghyeon Jeon

Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks (2024.emnlp-main)

Challenge:	Large language models (LLMs) have been widely used for relevance assessment in information retrieval, but maintaining and updating such models is resource-intensive, limiting their feasibility in dynamic and multilingual search environments.
Approach:	They propose to combine a generative SLM with an embedding-based SLM to achieve higher relevance judgment accuracy while reducing computational costs.
Outcome:	The proposed approach outperforms state-of-the-art LLMs in relevance assessment tasks while reducing computational costs.

Copied to clipboard

Haoyuan Wu, Haisheng Zheng, Zhuolun He, Bei Yu

Streamlining LLMs: Adaptive Knowledge Distillation for Tailored Language Models (2025.naacl-srw)

Challenge:	Large language models (LLMs) have demonstrated considerable proficiency in general natural language processing tasks.
Approach:	They propose a parameter-efficient sparsity crafting method which crafts dense models into sparse models using the mixture-of-experts architecture.
Outcome:	The proposed method significantly reduces computational costs and GPU memory requirements, while maintaining the quality of approximation in function space.

Copied to clipboard

Prajvi Saxena, Sabine Janzen, Wolfgang Maass

Does Self-Attention Need Separate Weights in Transformers? (2025.naacl-industry)

Challenge:	Large language models (LLMs) have transformative potential across industries, e.g., enhancing customer service, revolutionizing medical diagnostics, or identifying crises in news articles.
Approach:	They propose to distill compact, parameter-efficient tailored language models from LLMs for domain-specific tasks with comparable performance.
Outcome:	The proposed framework outperforms knowledge distillation frameworks in the crisis domain, where labeled data is scarce.

Copied to clipboard

Md Kowsher, Nusrat Jahan Prottasha, Chun-Nam Yu, Ozlem Garibay, Niloofar Yousefi

AutoML Meets Hugging Face: Domain-Aware Pretrained Model Selection for Text Classification (2025.naacl-srw)

Challenge:	Experimental results show a 66.53% reduction in parameter size within the attention block and competitive accuracy improvements of 3.55% and 0.89% over symmetric and pairwise attention-based models, respectively.
Approach:	They propose a simplified approach where a single weight matrix is used for Keys, Queries, and Values instead of separate matrices for each.
Outcome:	The proposed approach outperforms the BERT baseline on GLUE tasks even outperforming the standard BERT model in handling noisy and out-of-domain data.

Copied to clipboard

Parisa Safikhani, David Broneske

OpenBioNER: Lightweight Open-Domain Biomedical Named Entity Recognition Through Entity Type Description (2025.findings-naacl)

Challenge:	Existing methods for fine-tuning text classification models are resource-intensive and require substantial computational power and time.
Approach:	They propose a corpus-driven domain mapping pipeline that integrates pre-fine-tuned models from Hugging Face Model Hub into AutoML systems to improve model selection.
Outcome:	The proposed pipeline improves model selection and streamlines workflows and reduces computational costs.

Copied to clipboard

Alessio Cocchieri, Giacomo Frisoni, Marcos Martínez Galindo, Gianluca Moro, Giuseppe Tagliavini, Francesco Candoli

ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models Reranking (2026.findings-acl)

Challenge:	Biomedical Named Entity Recognition (BioNER) is a computationally expensive and limited tool . specialized 7B NER LLMs and GPT-4o can't match textual spans with entity types .
Approach:	They propose a lightweight BERT-based cross-encoder architecture that can identify any biomedical entity using only its description.
Outcome:	The proposed system outperforms existing models that match textual spans with entity types rather than descriptions on biomedical benchmarks.

Copied to clipboard

Xianming LI, Aamir Shakir, Rui Huang, Julius Lipp, Benjamin Clavié, Jing Li

Make Large Language Model a Better Ranker (2024.findings-emnlp)

Challenge:	Recent Large Language Models (LLMs) have demonstrated remarkable performance in document reranking tasks.
Approach:	They propose a two-stage training approach for document reranking using reinforcement learning and fine-grained score learning.
Outcome:	The proposed approach outperforms open-source and proprietary reranking models on BEIR benchmark.

Copied to clipboard

Wen-Shuo Chao, Zhi Zheng, Hengshu Zhu, Hao Liu

CoreGaze: Core Subgraph-Driven Visual Gaze Diffusion for Training-Free Referring Multimodal Large Language Models (2026.acl-long)

Challenge:	Large Language Models (LLMs) demonstrate robust capabilities across various fields . current list-wise approaches fail in ranking tasks due to misalignment between ranking objectives and next-token prediction .
Approach:	They propose a large language model framework with Aligned Listwise Ranking Objectives (ALRO) this framework provides explicit feedback in a listwise manner by introducing soft lambda loss .
Outcome:	The proposed model outperforms existing recommendation methods and embedding-based recommendations without additional computational burdens.

Copied to clipboard

Xiaoyang Yi, Jing Chen, Yuru Bao, Jian Zhang

SLENDER: Structured Outputs for SLM-based NER in Low-Resource Englishes (2025.acl-industry)

Challenge:	Existing methods rely on extensive fine-tuning to mitigate attention distraction, leading to redundant outputs or hallucinations.
Approach:	They propose a training-free framework that simulates human visual gaze diffusion for fine-grained comprehension by combining a sparse semantic graph with a core subgraph with amplified initial influence.
Outcome:	The proposed framework simulates human visual gaze diffusion for fine-grained comprehension.

Copied to clipboard

Nicole Ren, James Teo

Better Late Than Never: Model-Agnostic Hallucination Post-Processing Framework Towards Clinical Text Summarization (2024.findings-acl)

Challenge:	Named Entity Recognition (NER) for low-resource variants of English remains challenging, as most models are trained on datasets predominantly focused on American or British English.
Approach:	They propose a new output format for Named Entity Recognition (NER) that achieves a three-fold reduction in inference time compared to JSON format.
Outcome:	The proposed output format achieves a three-fold reduction in inference time on average compared to JSON format, which is widely used for structured outputs.

Copied to clipboard

Songda Li, Yunqi Zhang, Chunyuan Deng, Yake Niu, Hui Zhao

PLEX: Adaptive Parameter-Efficient Fine-Tuning for Code LLMs using Lottery-Tickets (2025.naacl-industry)

Challenge:	Existing methods for generating concise and coherent summaries may include unintended text with hallucinations, causing computational costs.
Approach:	They propose a model-agnostic framework to post-process medical hallucinations . MEDAL integrates with any medical summarization model, requiring no additional computational overhead .
Outcome:	MEDAL can post-process medical hallucinations without additional computational overhead.

Copied to clipboard

Jaeseong Lee, Hojae Han, Jongyoon Kim, Seung-won Hwang, Naun Kang, KyungJun An, Sungho Jang

The LLM Already Knows: Estimating LLM-Perceived Question Difficulty via Hidden Representations (2025.emnlp-main)

Challenge:	PLEX is a lottery-ticket based parameter-efficient fine-tuning method that adapts large language models to well-supported and underrepresented programming languages (PLs) in pretraining.
Approach:	They propose a lottery-ticket based parameter-efficient fine-tuning method that adapts large language models to well-supported and underrepresented programming languages (PLs)
Outcome:	The proposed method achieves state-of-the-art performance among PEFT methods while maintaining competitive results with reduced computational overhead.

Copied to clipboard

Yubo Zhu, Dongrui Liu, Zecheng Lin, Wei Tong, Sheng Zhong, Jing Shao

SCV: Light and Effective Multi-Vector Retrieval with Sequence Compressive Vectors (2025.coling-industry)

Challenge:	Existing methods for difficulty estimation rely on repeated response sampling, auxiliary models, or fine-tuning the target model itself.
Approach:	They propose a method that leverages only the hidden representations produced by large language models.
Outcome:	The proposed method outperforms baselines in difficulty estimation on textual and multimodal tasks and improves adaptive reasoning strategies with fewer generated tokens.

Copied to clipboard

Cheoneum Park, Seohyeong Jeong, Minsang Kim, KyungTae Lim, Yong-Hun Lee

Efficient Vocabulary Reduction for Small Language Models (2025.coling-industry)

Challenge:	Recent advances in language models (LMs) have driven progress in information retrieval (IR), effectively extracting semantically relevant information.
Approach:	They propose a multi-vector retrieval framework with sequence compression vectors and a coarse-to-fine vector search framework that uses span compressive vectors to score documents.
Outcome:	The proposed approach achieves the fastest latency compared to other state-of-the-art models and can obtain competitive performance on both in-domain and out-of domain benchmark datasets.

Copied to clipboard

Yuta Nozaki, Dai Nakashima, Ryo Sato, Naoki Asaba

DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models (2024.acl-long)

Challenge:	Large language models (LLMs) have high computational costs and energy consumption, making their deployment in industrial settings difficult.
Approach:	They propose a small language model that compresses the embedding layer and reduces model size without significant loss of performance.
Outcome:	The proposed model reduces the embedding layer while maintaining performance while improving accuracy and performance.

Copied to clipboard

Damai Dai, Chengqi Deng, Chenggang Zhao, R.x. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y.k. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, Wenfeng Liang

SQLGenie: A Practical LLM based System for Reliable and Efficient SQL Generation (2025.acl-industry)

Challenge:	Mixture-of-Experts (MoE) architectures face challenges in ensuring expert specialization . despite the promising performance, scaling language models to an extremely large scale is associated with exceedingly high computational costs.
Approach:	They propose an architecture that allows for ultimate expert specialization by segmenting experts into mN ones and activating mK from them.
Outcome:	The proposed architecture achieves comparable performance with GShard with 2B parameters and computation.

Copied to clipboard

Pushpendu Ghosh, Aryan Jain, Promod Yenigalla

Hemolix.TabGen: Optimized Table Generation from Documents (2026.acl-industry)

Challenge:	Large Language Models (LLMs) enable natural language to SQL conversion, but generating accurate, efficient queries is challenging due to ambiguous intent, domain knowledge requirements and database constraints.
Approach:	They propose a system for reliable SQL generation that integrates Table Onboarder, SQL Generator and Feedback Augmentation.
Outcome:	The proposed system surpasses the best single-LLM baseline by 21.5% and the strongest pipeline competitor by 5.3% on public benchmarks and internal datasets.

Copied to clipboard

Gyanendra Shrestha, Todor Ivanov, Karthik Vemireddy, Anna Pyayt, Michael Gubanov

On Sample-Efficient Code Generation (2023.emnlp-industry)

Challenge:	Modern Data Lakes contain vast and heterogeneous document collections, making table generation difficult.
Approach:	They propose a scalable LLM-based table generation system that comprehends documents and generates Bi-dimensional tables based on the entire document content.
Outcome:	The proposed system generates bi-dimensional tables from documents across multiple domains.

Copied to clipboard

Hojae Han, Yu Jin Kim, Byoungjip Kim, Youngwon Lee, Kyungjae Lee, Kyungmin Lee, Moontae Lee, Kyunghoon Bae, Seung-won Hwang

Programming Languages in NLP

Challenge:	Existing approaches to code generation rely on rejection sampling to generate multiple code snippets then select the best.
Approach:	They propose a framework that prioritizes sampling on test problems that models can solve.
Outcome:	The proposed framework reduces sampling costs while maintaining comparable code generation performance.

Too Late to Train, Too Early To Use? A Study on Necessity and Viability of Low-Resource Bengali LLMs (2025.coling-main)

Copied to clipboard

Tamzeed Mahfuz, Satak Kumar Dey, Ruwad Naswan, Hasnaen Adil, Khondker Salman Sayeed, Haz Sameen Shahgir

OrchestraLLM: Efficient Orchestration of Language Models for Dialogue State Tracking (2024.naacl-long)

Challenge:	a new generation of English-oriented Large Language Models significantly outperforms older LLMs on low-resource languages.
Approach:	They compare Bengali-oriented LLMs with open-weight and closed-source LLM models . they conclude that there is a need for a Bengali model, but lacks high-quality pretraining data .
Outcome:	The proposed model outperforms existing models on Bengali on low-resource languages . the results highlight biases in machine-translated datasets used for Bengali NLP tasks .

Copied to clipboard

Chia-Hsuan Lee, Hao Cheng, Mari Ostendorf

Invoke Interfaces Only When Needed: Adaptive Invocation for Large Language Models in Question Answering (2025.findings-emnlp)

Challenge:	Large language models (LLMs) are computationally expensive and often require computational resources.
Approach:	They propose a routing framework that seamlessly integrates a SLM and an LLM, or-lm, or a LLM into a single framework.
Outcome:	The proposed routing framework reduces the computational costs by over 50% in dialogue state tracking tasks.

Copied to clipboard

Jihao Zhao, Chunlai Zhou, Daixuan Li, Shuaishuai Zu, Biao Qin

TELLME: Test-Enhanced Learning for Language Model Enrichment (2026.findings-eacl)

Challenge:	a new metric is developed to pinpoint the moment of invocation when hallucinations arise in small LMs.
Approach:	They propose a metric that measures hallucinations during the generation process of small LMs.
Outcome:	The proposed metric outperforms baselines in hallucination detection across multiple QA datasets.

Copied to clipboard

Minjun Kim, Inho Won, HyeonSeok Lim, MinKyu Kim, Junghun Yuk, Wooyoung Go, Jongyoul Park, Jungyeul Park, KyungTae Lim

Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inference (2026.findings-eacl)

Challenge:	Continual pre-training (CPT) has been widely adopted as a method for domain expansion in large language models, but has faced challenges such as acquiring large-scale domain-specific datasets and high computational costs.
Approach:	They propose a method that integrates the Test-Enhanced Learning principle with CPT to promote efficient domain-specific knowledge acquisition and long-term memory retention.
Outcome:	The proposed method outperforms existing methods by 23.6% in the financial domain and achieves 9.8% improvement in long-term memory retention.

Copied to clipboard

Bo-Wei Chen, Chung-Chi Chen, An-Zi Yen

Transcending Scaling Laws with 0.1% Extra Compute (2023.emnlp-main)

Challenge:	Large Language Models (LLMs) have revolutionized inference across diverse natural language tasks, with larger models performing better but at higher computational costs.
Approach:	They propose a confidence-driven strategy that dynamically selects the most suitable model based on confidence estimates.
Outcome:	The proposed approach reduces token usage by approximately 60% and improves cost efficiency on the Massive Multitask Language Understanding (MMLU) benchmark.

Copied to clipboard

Yi Tay, Jason Wei, Hyung Chung, Vinh Tran, David So, Siamak Shakeri, Xavier Garcia, Steven Zheng, Jinfeng Rao, Aakanksha Chowdhery, Denny Zhou, Donald Metzler, Slav Petrov, Neil Houlsby, Quoc Le, Mostafa Dehghani

SELF-GUARD: Empower the LLM to Safeguard Itself (2024.naacl-long)

Challenge:	Existing scaling of language models is expensive and requires significant computational costs.
Approach:	They propose a method that substantially improves existing language models and their scaling curves with a relatively tiny amount of extra compute.
Outcome:	The proposed method significantly improves existing language models and their scaling curves with a relatively tiny amount of extra compute.

Copied to clipboard

Zezhong Wang, Fangkai Yang, Lu Wang, Pu Zhao, Hongru Wang, Liang Chen, Qingwei Lin, Kam-Fai Wong

L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models (2025.acl-long)

Challenge:	Recent studies have investigated methods to improve the safety of large language models (LLMs) safety training involves fine-tuning the LLM with adversarial samples, which activate the LRM’s capabilities against jailbreak.
Approach:	They propose a safety training approach that integrates safety training and safeguards to train the LLM to perform harmfulness detection on its own outputs.
Outcome:	The proposed method reduces harmful output and adds a [harmful] or [harmless] tag to the end of the LLM's response.

Copied to clipboard

Hyesung Jeon, Yulhwa Kim, Jae-Joon Kim

Revisiting OPRO: The Limitations of Small-Scale LLMs as Optimizers (2024.findings-acl)

Challenge:	Quantization-aware PEFT methods have been developed to reduce memory and computational costs associated with large language models.
Approach:	They propose a method that integrates Quantization-Aware Training (QAT) with LoRA to reduce memory overhead and improve model accuracy.
Outcome:	The proposed method significantly reduces QAT’s memory overhead while preserving the advantage of QAT in producing fully quantized LLMs with high accuracy.

Copied to clipboard

Tuo Zhang, Jinyue Yuan, Salman Avestimehr

GEMMAS: Graph-based Evaluation Metrics for Multi Agent Systems (2025.emnlp-industry)

Challenge:	Recent studies aim to enhance the efficacy of Large Language Models (LLMs) through strategic prompting.
Approach:	They propose to revisit the optimization by prompting approach for small-scale LLMs . they suggest future prompting engineering to consider both model capabilities and computational costs .
Outcome:	The proposed approach shows limited effectiveness in small-scale LLMs, with limited inference capabilities constraining optimization ability.

Copied to clipboard

Jisoo Lee, Raeyoung Chang, Dongwook Kwon, Harmanpreet Singh, Nikhil Verma

ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models (2025.findings-naacl)

Challenge:	Existing evaluations focus on the correctness of the final output, overlooking inefficient communication and poor coordination contribute to redundant reasoning and higher computational costs.
Approach:	They propose a graph-based evaluation framework that analyzes the internal collaboration process by modeling agent interactions as a directed acyclic graph.
Outcome:	The proposed framework shows that outcome-only metrics are insufficient for evaluating multi-agent performance on GSM8K.

Copied to clipboard

David Anugraha, Genta Indra Winata, Chenyue Li, Patrick Amadeus Irawan, En-Shiun Annie Lee

A Cost-Efficient Modular Sieve for Extracting Product Information from Company Websites (2024.emnlp-industry)

Challenge:	Performance prediction is a method to estimate the performance of Language Models (LMs) on various Natural Language Processing (NLP) tasks.
Approach:	They propose a task- and language-agnostic framework to predict the performance of Language Models (LMs) using proxy models.
Outcome:	The proposed framework outperforms the state-of-the-art in root-mean-square error (RMSE) and other robustness tests on multilingual NLP tasks.

Copied to clipboard

Anna Hätty, Dragan Milchevski, Kersten Döring, Marko Putnikovic, Mohsen Mesgar, Filip Novović, Maximilian Braun, Karina Borimann, Igor Stranjanac

Is Semantic Chunking Worth the Computational Cost? (2025.findings-naacl)

Challenge:	Existing methods for extracting product information are resource-intensive and computationally prohibitive due to website structure differences and numerous non-product pages.
Approach:	They propose a modular method that leverages low-cost classification models to filter out company web pages.
Outcome:	The proposed method improves on a new dataset of 7000 product and non-product web pages and reduces computational time and costs.

Copied to clipboard

Renyi Qu, Ruixuan Tu, Forrest Sheng Bao

Mixture of Diverse Size Experts (2024.emnlp-industry)

Challenge:	Recent advances in Retrieval-Augmented Generation (RAG) systems have popularized semantic chunking.
Approach:	They evaluate the effectiveness of semantic chunking using three common retrieval tasks . they find that the computational costs associated with semantic chunks are not justified by consistent performance gains.
Outcome:	The proposed semantic chunking approach is not able to deliver consistent performance gains in three retrieval-related tasks.

Copied to clipboard

Manxi Sun, Wei Liu, Jian Luan, Pengzhi Gao, Bin Wang

Challenge:	Recent large language models (LLMs) have shown superior performance in a variety of tasks due to the sub-linearly increasing computational costs.
Approach:	They propose a new MoE architecture with designed layers where experts have different sizes to mitigate this defect.
Outcome:	The proposed architecture surpasses existing MoEs by adaptively assigning the parameter budget to experts while maintaining the same total parameter size and number of experts.

Rethinking Scale: Deployment Trade-offs of Small Language Models under Agent Paradigms (2026.acl-industry)

Copied to clipboard

Xinlin Wang

How Accurate Are LLMs at Multi-Question Answering on Conversational Transcripts? (2025.emnlp-industry)

Challenge:	Existing research focuses on enhancing large language models through scaling laws or fine-tuning strategies, but ignores the potential of using agent paradigms to compensate for the inherent weaknesses of small models.
Approach:	They propose to use structured agent frameworks to improve effectiveness over direct prompting . they also propose to employ routing-based multi-agent systems with collaborative capabilities .
Outcome:	The proposed model significantly outperforms direct prompting with single-agent systems . the proposed model is more reliable and cost-effective than other models .

Copied to clipboard

Xiliang Zhu, Shi Zong, David Rossouw

Natural Language Interfaces

Challenge:	Large Language Models (LLMs) are used for question answering over long contexts . high computational costs and latency hinder the process .
Approach:	They explore the capabilities of Large Language Models to answer multiple questions based on the same conversational context.
Outcome:	The proposed models outperform proprietary and public models in question answering . their results show that they can be cost-effective and transparent .

Learning to Paraphrase for Alignment with LLM Preference (2024.findings-emnlp)

Copied to clipboard

Junbo Fu, Guoshuai Zhao, Yimin Deng, Yunqi Mi, Xueming Qian

Beyond Examples: Towards Automated Thought-level In-Context Reasoning for Large Language Models (2026.acl-long)

Challenge:	Large Language Models (LLMs) exhibit the issue of paraphrase divergence, which means that when a question is phrased in a slightly different but semantically similar way, LLM may output a wrong response . retraining faces challenges in meeting the computational costs and privacy security demands of LLMs.
Approach:	They propose a black-box method that enhances model performance by paraphrasing questions in expressions preferred by the model.
Outcome:	The proposed method improves performance by paraphrasing questions in expressions preferred by the model.

Copied to clipboard

Jinyang Wu, Mingkuan Feng, Shuai Zhang, Feihu Che, Zhengqi Wen, Chonghua Liao, Ling Yang, Haoran Luo, Zheng Lian, Jianhua Tao

ALBA: Adaptive Language-Based Assessments for Mental Health (2024.naacl-long)

Challenge:	In-context learning (ICL) struggles with complex reasoning due to superficial, example-level implicit imitation.
Approach:	They propose an automated method that shifts from surface-level examples to more guidance-oriented thought patterns.
Outcome:	The proposed method achieves 80.6% accuracy on MATH and 62.5% on AMC, surpassing GPT-4o’s 77.2% and 57.5% accuracy.

Copied to clipboard

Vasudha Varadarajan, Sverker Sikström, Oscar Kjell, H. Schwartz

Linguistic Theories

Challenge:	Adaptive language-based assessments require a substantial sample of words per person for accuracy.
Approach:	They propose an adaptive language-based assessment task that involves ordering questions and scoring latent psychological trait using limited language responses to previous questions.
Outcome:	The proposed methods improve over non-adaptive baselines, but are more accurate and scalable with fewer questions.

LLMInit: A Free Lunch from Large Language Models for Selective Initialization of Recommendation (2025.emnlp-industry)

Copied to clipboard

Weizhi Zhang, Liangwei Yang, Wooseong Yang, Henry Peng Zou, Yuqing Liu, Ke Xu, Sourav Medya, Philip S. Yu

When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants (2023.acl-short)

Challenge:	Existing algorithms for collaborative filtering are limited by their computational demands and latency.
Approach:	They propose a framework to integrate pre-trained LLM embeddings into CF models through selective initialization strategies.
Outcome:	The proposed framework improves recommendation performance while maintaining low computational costs.

Copied to clipboard

Anuj Diwan, Eunsol Choi, David Harwath

Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Data (2025.findings-naacl)

Challenge:	Existing models focus on improving the efficiency of self-attention, but in practice they may be slower, especially given modest input lengths that are typical of many tasks.
Approach:	They propose a novel local-attention variant of a self-supervised speech model that uses input length thresholds to identify bottlenecks.
Outcome:	The proposed model is based on a self-attention-based model with a high input length threshold.

Copied to clipboard

Juanhui Li, Sreyashi Nag, Hui Liu, Xianfeng Tang, Sheikh Muhammad Sarwar, Limeng Cui, Hansu Gu, Suhang Wang, Qi He, Jiliang Tang

G-SPEED: General SParse Efficient Editing MoDel (2023.findings-emnlp)

Challenge:	Large Language Models (LLMs) have demonstrated superior language understanding abilities in many real-world NLP applications.
Approach:	They propose a learning-based sample selection method that incorporates signals from both teacher and student to enhance model performance.
Outcome:	The proposed method improves model performance across datasets with higher data efficiency.

Copied to clipboard

Haoke Zhang, Yue Wang, Juntao Li, Xiabing Zhou, Min Zhang

LSDC: An Efficient and Effective Large-Scale Data Compression Method for Supervised Fine-tuning of Large Language Models (2025.findings-naacl)

Challenge:	Large Language Models (LLMs) have demonstrated incredible capabilities in understanding, generating, and manipulating languages.
Approach:	They propose a general SParse Efficient Editing MoDel which can fulfill diverse editing requirements through a single model while maintaining low computational costs.
Outcome:	The proposed model can fulfill diverse editing requirements through a single model while maintaining low computational costs.

Copied to clipboard

Zhaoguang Long, Yuhao Zhou, Shangqing Zhao, Yupei Ren, Li Cai, Chenghao Jia, Zhe Chen, Zhe Fang, Yuxiang Song, Man Lan

Modality-specific Learning Rates for Effective Multimodal Additive Late-fusion (2022.findings-acl)

Challenge:	Large Language Models (LLMs) are expanding in scale and size, increasing computational costs . large-scale data compression techniques can reduce the size of training datasets while maintaining data integrity.
Approach:	They propose a large-scale data compression method to reduce the size of training data . they use a bifurcated quantization strategy to maximize the diversity of samples .
Outcome:	The proposed method significantly reduces the size of training data while maximizing the submodular gain.

Copied to clipboard

Yiqun Yao, Rada Mihalcea

Evolutionary Guided Decoding: Iterative Value Refinement for LLMs (2026.acl-long)

Challenge:	Multimodal machine learning uses additive late-fusion to combine feature representations from different modalities into a joint representation.
Approach:	They propose a Modality-Specific Learning Rate method to build late-fusion multimodal models from fine-tuned unimodal models.
Outcome:	The proposed method outperforms global learning rates on multiple tasks and settings and enables the models to effectively learn each modality.

Copied to clipboard

Zhenhua Liu, Lijun Li, Ruizhe Chen, Yuxian Jiang, Tong Zhu, Zhaochen Su, Wenliang Chen, Jing Shao

CRAFT: Training-Free Cascaded Retrieval for Tabular QA (2026.acl-long)

Challenge:	Existing methods for directing language model outputs are limited in their accuracy due to a distributional gap . existing methods train static value functions on trajectories sampled exclusively from the base policy .
Approach:	They propose a framework to bridge a distributional gap in the accuracy of value functions . they propose RLHF to align language models with human values and task requirements .
Outcome:	The proposed framework reduces computational costs and improves value function accuracy by leveraging principled value function optimization.

Copied to clipboard

Adarsh Singh, Kushal Raj Bhandari, Jianxi Gao, Soham Dan, Vivek Gupta

Meta-Learning of Prompt Generation for Lightweight Prompt Engineering on Language-Model-as-a-Service (2023.findings-emnlp)

Challenge:	Existing methods for open-domain table question answering require retraining or fine-tuning on new datasets.
Approach:	They propose a zero-shot, cascaded retrieval approach that uses a sparse retrieval model to filter a subset of candidates before applying more expensive dense models as re-rankers.
Outcome:	The proposed method outperforms state-of-the-art retrieval models on the NQ-Tables dataset.

Copied to clipboard

Hyeonmin Ha, Jihye Lee, Wookje Han, Byung-Gon Chun

Talk to Your Slides: High-Efficiency Slide Editing via Language-Driven Structured Data Manipulation (2026.findings-acl)

Challenge:	Language-Model-as-a-Services (LMaaSs) support a variety of user tasks through in-context learning from prompts.
Approach:	They propose a lightweight automatic prompt generation method that meta-trains a prompt generation model to enable robust learning from the contexts created by the generated prompts.
Outcome:	The proposed method improves performance on unseen tasks by 19.4% compared to the state-of-the-art prompt generation method.

Copied to clipboard

Kyudan Jung, Hojun Cho, Jooyeol Yun, Soyoung Yang, Jaehyeok Jang, Jaegul Choo

DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models (2023.emnlp-main)

Challenge:	Talk-to-Your-Slides is a high-efficiency slide editing agent that uses language-driven structured data manipulation instead of the image modality.
Approach:	They propose a language-driven slide editing agent that uses language-based structured data manipulation instead of image modality.
Outcome:	The proposed system achieves faster processing and better instruction fidelity than GUI-based agents.

Copied to clipboard

Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, Deyi Xiong

ModularMoE: Fast LLM Customization with Parameter-Sharing Mixture-of-Experts for Low-Resource Settings (2026.findings-acl)

Challenge:	Existing studies have demonstrated that pretrained language models memorize and regurgitate a significant portion of training data, including atypical data points that appear only once in the training data.
Approach:	They propose a method to locate and erase risky neurons in order to eliminate the impact of privacy data in the model in batches.
Outcome:	The proposed method eliminates the impact of privacy data in the model in batches without affecting the model's performance.

Copied to clipboard

Jiaxing Liu, Qi Qi, Haifeng Sun, Dunjun Li, Zirui Zhuang, Bo He, Xiang Yang, Cong Liu, Jianxin Liao, Jingyu Wang

Hierarchical Document Refinement for Long-context Retrieval-augmented Generation (2025.acl-long)

Challenge:	Large Language Models impose significant computational and storage burdens on personal devices . existing customization approaches incur excessive computational costs or lead to suboptimal performance .
Approach:	They propose a training framework that converts pre-trained LLMs into parameter-sharing MoE models for lightweight deployment.
Outcome:	The proposed training framework outperforms state-of-the-art training frameworks at the same sparsity level while delivering up to 2.71 inference speedup.

Copied to clipboard

Jiajie Jin, Xiaoxi Li, Guanting Dong, Yuyao Zhang, Yutao Zhu, Yongkang Wu, Zhonghua Li, Ye Qi, Zhicheng Dou

LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning (2024.findings-acl)

Challenge:	Real-world RAG applications often encounter long-context input scenarios where redundant information and noise results in higher inference costs and reduced performance.
Approach:	They propose an efficient plug-and-play refiner that leverages the structural characteristics of long documents.
Outcome:	Experiments on seven QA datasets show that LongRefiner achieves competitive performance in various scenarios while using 10x fewer computational costs and latency compared to baseline.

Copied to clipboard

Mingyang Zhang, Hao Chen, Chunhua Shen, Zhen Yang, Linlin Ou, Xinyi Yu, Bohan Zhuang

SEER: Self-Aligned Evidence Extraction for Retrieval-Augmented Generation (2024.emnlp-main)

Challenge:	Low-rank adaption (LoRA) is a low-level pruning method that can be expensive and slow to deploy.
Approach:	They propose a low-rank adaption pruning framework that provides an accurate structured pruned model in a memory-efficient manner.
Outcome:	The proposed pruning framework reduces perplexity and memory usage by 52.6% on LLaMA and T5 models while reducing memory usage.

Copied to clipboard

Xinping Zhao, Dongfang Li, Yan Zhong, Boren Hu, Yibin Chen, Baotian Hu, Min Zhang

TEACH: A Contrastive Knowledge Adaptive Distillation Framework for Classical Chinese Understanding (2025.acl-long)

Challenge:	Existing methods for enhancing RAG performance rely on heuristic-based augmentation . Existing approaches rely heavily on a heuriistic-driven approach, resulting in poor generalization and skews in the evidence length.
Approach:	They propose a model-based evidence extraction learning framework that optimizes a vanilla model as an evidence extractor with desired properties through self-aligned learning.
Outcome:	The proposed method reduces the evidence length by 9.25 times and improves reliability and reliability.

Copied to clipboard

Yuting Wei, Qi Meng, Yuanxing Xu, Bin Wu

Quality Assessment of Tabular Data using Large Language Models and Code Generation (2025.emnlp-industry)

Challenge:	Traditional methods for processing classical Chinese segment language understanding into discrete tasks, which overlook crucial background information and reduce user engagement.
Approach:	They propose a framework that integrates word sense disambiguation with sentence translation to minimize hallucinations and improve semantic analysis.
Outcome:	The proposed framework integrates word sense disambiguation with sentence translation to minimize hallucinations and improve semantic analysis.

Copied to clipboard

Ashlesha Akella, Akshar Kaul, Krishnasuri Narayanam, Sameep Mehta

Unlocking the Planning Capabilities of Large Language Models with Maximum Diversity Fine-tuning (2025.findings-naacl)

Challenge:	Data quality is vital for business decisions; poor data quality costs organizations an average of $12.9 million annually.
Approach:	They propose a framework that combines statistical inliner detection with LLM-driven rule and code generation.
Outcome:	The proposed framework produces semantically valid quality rules and validates them with retrieval-augmented generation (RAG) Extensive evaluations on benchmark datasets confirm the effectiveness of the proposed framework.

Copied to clipboard

Wenjun Li, Changyu Chen, Pradeep Varakantham

It’s All About the Confidence: An Unsupervised Approach for Multilingual Historical Entity Linking using Large Language Models (2026.eacl-long)

Challenge:	Existing studies have shown that LLMs struggle to generate valid plans in the automated planning domain due to weak System 2 competencies.
Approach:	They propose a method which uses a random sampling method to select diverse and representative data to enhance sample efficiency and the model’s generalization capability.
Outcome:	The proposed method outperforms baseline methods across scales and multiple benchmark domains.

Copied to clipboard

Cristian Santini, Marieke van Erp, Mehwish Alam

Query Optimization for Parametric Knowledge Refinement in Retrieval-Augmented Large Language Models (2025.findings-emnlp)

Challenge:	Existing approaches to EL for historical texts require substantial training data or rely on domain-specific rules that limit scalability.
Approach:	They propose an unsupervised ensemble approach combining a Small Language Model and an LLM for historical EL.
Outcome:	The proposed approach outperforms state-of-the-art models on four established benchmarks in six European languages from the 19th and 20th centuries.

Copied to clipboard

Youan Cong, Pritom Saha Akash, Cheng Wang, Kevin Chen-Chuan Chang

Icon2: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulation (2025.emnlp-main)

Challenge:	Extract-Refine-Retrieve-Read is a query optimization framework for large language models . it is designed to bridge the pre-retrieval information gap in Retriev-Augmented Generation systems .
Approach:	They propose a framework to extract parametric knowledge from Large Language Models and refine them using a specialized query optimizer.
Outcome:	The extract-refine-retrieve-read framework outperforms baselines on QA datasets . it is designed to meet the knowledge requirements of large language models (LLMs)

Copied to clipboard

Qiyuan Chen, Hongsen Huang, Qian Shao, Jiahe Chen, Jintai Chen, Hongxia Xu, Renjie Hua, Ren Chuan, Jian Wu

MeMoTune: A Measure and Moment-Driven Fine-Tuning Framework for Quantized Large Language Models (2025.findings-acl)

Challenge:	Large Language Models (LLMs) require high quality preference datasets to align with human preferences.
Approach:	They propose a framework that leverages inherent regulation of LLMs’ representation space for efficient and tailored preference dataset construction, named Icon2.
Outcome:	The proposed framework improves performance on benchmarks like AlpacaEval 2.0 and Arena-Hard while reducing computational costs by up to 48.1%.

Copied to clipboard

Yun Zhang, Xue Geng, Lizi Liao, Jintong Sun, Minghe Yu, Ge Yu

Adaptive Gating in Mixture-of-Experts based Language Models (2023.emnlp-main)

Challenge:	Existing methods combine quantization with parameter-efficient fine-tuning but fail to meet practical performance requirements.
Approach:	They propose a measure and moment approach to optimize objective function for superior fine-tuning results by scaling the update process through a gradient.
Outcome:	The proposed framework outperforms state-of-the-art methods on tasks like text generation, summarization, and understanding.

Copied to clipboard

Jiamin Li, Qiang Su, Yitao Yang, Yimin Jiang, Cong Wang, Hong Xu

Token-level Dynamic Self-Attention Network for Multi-Passage Reading Comprehension (P19-1)

Challenge:	Existing models employ a fixed gating network where each token is computed by the same number of experts.
Approach:	They propose a flexible training strategy that allows tokens to be processed by a variable number of experts based on expert probability distribution.
Outcome:	The proposed model reduces training time and inference quality while maintaining sparsity while maintaining inference accuracy.

Copied to clipboard

Yimeng Zhuang, Huadong Wang

Machine Reading Comprehension

Challenge:	Multi-passage reading comprehension requires the ability to combine cross-passages information and reason over multiple passages to infer the answer.
Approach:	They propose a Dynamic Self-attention Network (DynSAN) which processes cross-passage information at token-level and meanwhile avoids substantial computational costs.
Outcome:	The proposed model achieves state-of-the-art performance on the SearchQA, Quasar-T and WikiHop datasets and further ablation validates the effectiveness of its components.

Efficient Prompting for Continual Adaptation to Missing Modalities (2025.naacl-long)

Copied to clipboard

Zirun Guo, Shulei Wang, Wang Lin, Weicai Yan, Yangyang Wu, Tao Jin

Longtriever: a Pre-trained Long Text Encoder for Dense Document Retrieval (2023.emnlp-main)

Challenge:	Existing methods combine various missing cases to train recovery modules or align multimodal features, resulting in suboptimal performance, high computational costs, and catastrophic forgetting.
Approach:	They propose a continual multimodal missing modality task that uses prompts to learn modalities . existing methods often aggregate various missing cases to train recovery modules . authors conduct extensive experiments on three public datasets .
Outcome:	The proposed method consistently outperforms state-of-the-art methods on three public datasets.

Copied to clipboard

Junhan Yang, Zheng Liu, Chaozhuo Li, Guangzhong Sun, Xing Xie

Calibrating Beyond English: Language Diversity for Better Quantized Multilingual LLMs (2026.eacl-long)

Challenge:	Existing PLMs are infeasible for processing long documents due to computational costs and incomprehensive document understanding.
Approach:	They propose a retrieval model that models local semantics and global context semantics in a tightly-coupled manner.
Outcome:	The proposed model overcomes three core challenges of long document retrieval: substantial computational cost, incomprehensive document understanding, and scarce annotations.

Copied to clipboard

Everlyn Asiko Chimoto, Mostafa Elhoushi, Bruce Bassett

Unleashing the Potential of Large Language Models through Spectral Modulation (2024.findings-emnlp)

Challenge:	Existing quantization methods typically use small, English-only calibration sets . however, their impact on multilingual models remains underexplored .
Approach:	They evaluate eight calibration settings across two quantizers on data from 10 different languages.
Outcome:	The results show that tailoring calibration sets to the evaluation language yields the largest improvements for individual languages, underscoring the importance of linguistic alignment.

Copied to clipboard

Peng Sun, Yao Zhu, Yunjian Zhang, Xiu Yan, Zizhe Wang, Xiangyang Ji

Efficient and Effective Prompt Tuning via Prompt Decomposition and Compressed Outer Product (2025.naacl-long)

Challenge:	Large Language Models (LLMs) have demonstrated impressive capabilities across various domains, garnering significant attention from both academia and industry.
Approach:	They propose to conduct spectral modulation in the parameter space of LLMs to integrate with various models in a plug-and-play manner.
Outcome:	The proposed approach improves performance by 10.12% with spectral modulation.

Copied to clipboard

Pengxiang Lan, Haoyu Xu, Enneng Yang, Yuliang Liang, Guibing Guo, Jianzhe Zhao, Xingwei Wang

Comprehensive Layer-wise Analysis of SSL Models for Audio Deepfake Detection (2025.findings-naacl)

Challenge:	Existing methods for fine-tuning pre-trained language models overlook intrinsic semantic associations between soft prompt tokens, leading to high discreteness and limited interactions.
Approach:	They propose a low-parameters Prompt Tuning method which leverages prompt decomposition and compressed outer product to facilitate multiple interactions among prompt tokens.
Outcome:	Experiments on six architectures and eight datasets show that the proposed method outperforms state-of-the-art methods in performance and efficiency.

Copied to clipboard

Yassine El Kheir, Younes Samih, Suraj Maharjan, Tim Polzehl, Sebastian Möller

Language Model-Driven Data Pruning Enables Efficient Active Learning (2026.findings-eacl)

Challenge:	Existing algorithms for audio deepfake detection are based on layer-wise analysis of self-supervised learning (SSL) models.
Approach:	They conduct a layer-wise analysis of self-supervised learning (SSL) models for audio deepfake detection across diverse contexts.
Outcome:	The proposed models achieve competitive equal error rate (EER) scores even when employing a reduced number of layers.

Copied to clipboard

Abdul Hameed Azeemi, Ihsan Ayyub Qazi, Agha Ali Raza

AnRe: Analogical Replay for Temporal Knowledge Graph Forecasting (2025.acl-long)

Challenge:	Existing data pruning methods for active learning are expensive and time-consuming.
Approach:	They propose a plug-and-play data pruning strategy that leverages language models to prune the unlabeled pool.
Outcome:	The proposed pruning strategy outperforms existing pruning methods on translation, sentiment analysis, topic classification, and summarization tasks on diverse datasets.

Copied to clipboard

Guo Tang, Zheng Chu, Wenxiang Zheng, Junjia Xiang, Yizhuo Li, Weihao Zhang, Ming Liu, Bing Qin

Exploring Hybrid Sampling Inference for Aspect-based Sentiment Analysis (2025.findings-naacl)

Challenge:	Temporal Knowledge Graphs (TKGs) are vital for event prediction, yet current methods face limitations.
Approach:	They propose a training-free Analogical Replay reasoning framework that uses LLMs to extract historical contexts and generate analogical reasoning examples as contextual inputs.
Outcome:	The proposed model outperforms existing training-free methods on four benchmarks.

Copied to clipboard

Xiaoyi Bao, Minjie Qiang, Jinghang Gu, Zhongqing Wang, Chu-Ren Huang

Sentiment Analysis

Challenge:	Existing methods for inference require multiple sampling with preset size . however, it is a high-cost method that requires multiple sampling .
Approach:	They propose a method that combines multiple and single sampling to greatly reduce the cost of multiple sampling without sacrificing performance.
Outcome:	The proposed method greatly reduces the cost of multiple sampling without sacrificing performance.

BeSimulator: A Large Language Model Powered Text-based Behavior Simulator (2025.emnlp-main)

Copied to clipboard

Jianan Wang, Bin Li, Jingtao Qi, Xueying Wang, Fu Li, null Lihanxun

StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements (2024.emnlp-main)

Challenge:	Existing robot simulators focus on physical process modeling and realistic rendering, resulting in high computational costs and limited adaptability.
Approach:	They propose a modular and novel LLM-powered framework to analyze and validate robot behaviors in text-based environments.
Outcome:	The proposed framework can generalize across scenarios and achieve long-horizon complex simulation.

Copied to clipboard

Jillian Fisher, Skyler Hallinan, Ximing Lu, Mitchell Gordon, Zaid Harchaoui, Yejin Choi

BERT-EMD: Many-to-Many Layer Mapping for BERT Compression with Earth Mover’s Distance (2020.emnlp-main)

Challenge:	Authorship obfuscation methods that ignore author-specific stylistic features are often too rigid and lead to degradation of fluency and grammaticality.
Approach:	They propose an adaptive obfuscation method that perturbs stylistic elements of text . authors release a large set of 30K high-quality, long-form texts from a diverse set of 14 authors .
Outcome:	The proposed method outperforms state-of-the-art methods on an array of domains on automatic and human evaluation.

Copied to clipboard

Jianquan Li, Xiaokang Liu, Honghong Zhao, Ruifeng Xu, Min Yang, Yaohong Jin

Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs (2024.acl-long)

Challenge:	Pre-trained language models have been proposed and applied to many NLP tasks, yielding state-of-the-art performance, but high storage and computational costs obstruct them to be effectively deployed on resource-constrained devices and real-time applications.
Approach:	They propose a BERT distillation method which allows each intermediate student layer to learn from any intermediate teacher layers.
Outcome:	The proposed method can learn from different teacher layers adaptively for different NLP tasks.

Copied to clipboard

Jiejun Tan, Zhicheng Dou, Yutao Zhu, Peidong Guo, Kun Fang, Ji-Rong Wen

PromptReps: Prompting Large Language Models to Generate Dense and Sparse Representations for Zero-Shot Document Retrieval (2024.emnlp-main)

Challenge:	Existing methods to determine the knowledge an LLM already possesses and the knowledge that requires the help of a search engine are expensive and require excessive computational costs.
Approach:	They propose a slim proxy model that detects missing knowledge in LLMs with a proxy model and use it to perform retrieval for the missing knowledge.
Outcome:	The proposed approach detects missing knowledge in LLMs with a slim proxy model and takes its answers as heuristic answers.

Copied to clipboard

Shengyao Zhuang, Xueguang Ma, Bevan Koopman, Jimmy Lin, Guido Zuccon

RED: Unleashing Token-Level Rewards from Holistic Feedback via Reward Redistribution (2025.emnlp-main)

Challenge:	Large language models (LLMs) excel in zero-shot document ranking tasks.
Approach:	They propose a prompt-based re-ranking method that requires no further training but is only feasible for reranking a handful of candidates due to computational costs.
Outcome:	The proposed method can retrieve documents from the entire corpus without training and with a large amount of paired text data.

Copied to clipboard

Jiahui Li, Lin Li, Tai-Wei Chang, Kun Kuang, Long Chen, Jun Zhou, Cheng Yang

GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models (2025.acl-long)

Challenge:	Experimental results demonstrate the superiority of our approach to aligning large language models with human preferences.
Approach:	They propose a method that evaluates and assigns specific credit to each token using an off-the-shelf reward model.
Outcome:	The proposed method evaluates and assigns specific credit to each token using an off-the-shelf reward model.

Copied to clipboard

Kai Yao, Zhaorui Tan, Penglei Gao, Lichun Li, Kaixin Wu, Yinggui Wang, Yuan Zhao, Yixin Ji, Jianke Zhu, Wei Wang

Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation (2021.acl-long)

Challenge:	Existing methods for offsite-tuning of large language models require high computational costs and lack theoretical analysis.
Approach:	They propose an offsite-tuning approach that selectively applies compression techniques such as rank compression and channel pruning to preserve the gradients of fine-tuned adapters while ensuring privacy.
Outcome:	The proposed method surpasses existing OT methods in privacy protection and model performance.

Copied to clipboard

Shizhe Diao, Ruijia Xu, Hongjin Su, Yilei Jiang, Yan Song, Tong Zhang

Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment (2025.findings-acl)

Challenge:	Existing methods to train pre-trained models require domain-specific data and computational resources.
Approach:	They propose a domain-aware N-gram Adaptor to incorporate unseen and domain-specific words into a generic pretrained model.
Outcome:	The proposed model can improve on eight low-resource tasks using limited data with lower computational costs.

Copied to clipboard

Xiaotian Zhang, Ruizhe Chen, Yang Feng, Zuozhu Liu

DiPair: Fast and Accurate Distillation for Trillion-Scale Text Matching and Pair Modeling (2020.findings-emnlp)

Challenge:	Existing methods for aligning language models with human preferences rely on reward signals and additional annotated data, limiting their scalability and adaptability to diverse human values.
Approach:	They propose a discriminative paradigm that leverages the intrinsic preference judgment capabilities of the model to align language models with human preferences.
Outcome:	The proposed model is scalable and efficient, paving the way for more adaptive personalized alignment.

Copied to clipboard

Jiecao Chen, Liu Yang, Karthik Raman, Michael Bendersky, Jung-Jung Yeh, Yun Zhou, Marc Najork, Danyang Cai, Ehsan Emadzadeh

Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translation (2025.findings-acl)

Challenge:	Existing knowledge distillation models are not optimized for dealing with pairs (or tuples) of texts.
Approach:	They propose a framework for distilling fast and accurate models on text pair tasks using a scalable end-to-end training strategy.
Outcome:	Empirical studies on academic and real-world e-commerce benchmarks show the proposed framework can achieve speedups of over 350x and minimal quality drop relative to the cross-attention teacher BERT model.

Copied to clipboard

Zhanglin Wu, Daimeng Wei, Xiaoyu Chen, Hengchao Shang, Jiaxin Guo, Zongyao Li, Yuanchang Luo, Jinlong Yang, Zhiqiang Rao, Hao Yang

Jailbreaking with Universal Multi-Prompts (2025.findings-naacl)

Challenge:	Large language models have advantages over neural machine translation systems, but they suffer from high computational costs and significant latency.
Approach:	They propose a scheduling policy that optimizes translation result while ensuring fast speed and as little LLM usage as possible.
Outcome:	The proposed model achieves optimal translation performance with less LLM usage on multilingual test sets.

Copied to clipboard

Yu-Ling Hsu, Hsuan Su, Shang-Tse Chen

HadSkip: Homotopic and Adaptive Layer Skipping of Pre-trained Language Models for Efficient Inference (2023.findings-emnlp)

Challenge:	Large language models (LLMs) have seen rapid development in recent years, but ethical concerns and new types of attacks have emerged.
Approach:	They propose a prompt-based method to jailbreak large language models using universal multi-prompts and an approach for defense that outperforms existing techniques.
Outcome:	The proposed method outperforms existing techniques for jailbreaking LLMs using universal multi-prompts.

Copied to clipboard

Haoyu Wang, Yaqing Wang, Tianci Liu, Tuo Zhao, Jing Gao

EcoLANG: Efficient and Effective Agent Communication Language Induction for Social Simulation (2025.findings-emnlp)

Challenge:	Existing methods to exit pre-trained language models suffer from the limitation that they have to sequentially traverse through all layers prior to the selected exit layer, which degrades their performance.
Approach:	They propose a homotopic and adaptive layer skipping fine-tuning method that adaptively selects the layers to skip based on a predefined budget.
Outcome:	The proposed method outperforms all state-of-the-art baselines on the GLUE benchmark and shows that it is highly efficient.

Copied to clipboard

Xinyi Mou, Chen Qian, Wei Liu, Ling Yan, Yao Hu, Xuanjing Huang, Zhongyu Wei

Instruction Tuning with Retrieval-based Examples Ranking for Aspect-based Sentiment Analysis (2024.findings-acl)

Challenge:	Large language models (LLMs) have demonstrated impressive ability to role-play humans and replicate complex social dynamics.
Approach:	They propose an efficient agent communication language induction for social simulations that reduces token consumption by over 20%.
Outcome:	The proposed model reduces token consumption by over 20% while preserving human language.

Copied to clipboard

Guangmin Zheng, Jin Wang, Liang-Chih Yu, Xuejie Zhang

Sentiment Analysis

Challenge:	Aspect-based sentiment analysis (ABSA) identifies sentiment information related to specific aspects . previous studies have proposed using fixed examples for instruction tuning .
Approach:	They propose an instruction learning method with retrieval-based example ranking for ABSA tasks.
Outcome:	The proposed method is superior to existing models on three ABSA subtasks.

MaPPER: Multimodal Prior-guided Parameter Efficient Tuning for Referring Expression Comprehension (2024.emnlp-main)

Copied to clipboard

Ting Liu, Zunnan Xu, Yue Hu, Liangtao Shi, Zhiqiang Wang, Quanjun Yin

POP: Prefill-Only Pruning for Efficient Large Model Inference (2026.findings-acl)

Challenge:	Existing methods for Referring Expression Comprehension (REC) lack specific domain abilities for precise local visual perception and visual-language alignment.
Approach:	They propose a framework for Parameter-Efficient Transfer Learning to localize a visual region via natural language using a prior-guided prior.
Outcome:	The proposed framework achieves the best accuracy compared to the current methods with only 1.41% tunable backbone parameters.

Copied to clipboard

Junhui He, Zhihui Fu, Jun Wang, Qingan Li

AQuilt: Weaving Logic and Self-Inspection into Low-Cost, High-Relevance Data Synthesis for Specialist LLMs (2025.emnlp-main)

Challenge:	Existing structured pruning methods suffer from significant accuracy degradation . Existing pruning methods are expensive and require specialized hardware and kernels to perform .
Approach:	They propose a stage-agnostic pruning approach that overlooks asymmetric roles between prefill and decode stages.
Outcome:	The proposed pruning approach achieves 1.37 speedup in prefill latency with minimal performance loss.

Copied to clipboard

Xiaopeng Ke, Hexuan Deng, Xuebo Liu, Jun Rao, Zhenxi Song, Jun Yu, Min Zhang

Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning (2025.coling-main)

Challenge:	Existing approaches to synthesis large language models often suffer from performance limitations and high computational costs.
Approach:	They propose a framework for constructing instruction-tuning data from unlabeled data for any specialized domains from corresponding unlabed data.
Outcome:	The proposed framework is comparable to DeepSeek-V3 while utilizing just 17% of the production cost.

Copied to clipboard

Xiaoye Qu, Jiashuo Sun, Wei Wei, Daizong Liu, Jianfeng Dong, Yu Cheng

Extending Context Window of Large Language Models via Semantic Compression (2024.findings-acl)

Challenge:	Large Vision-Language Models (LVLMs) have impressive capabilities in multi-modal context comprehension, but they still suffer from hallucination problems due to inconsistent outputs with the image content.
Approach:	They propose a training-free framework MVP to reduce hallucinations in Large Vision-Language Models . they propose multi-view information-seeking strategy to perceive the comprehensive information in the image .
Outcome:	The proposed framework reduces hallucinations in large vision-language models by combining multi-view multi-path reasoning with multi-vision multi-path reasoning.

Copied to clipboard

Weizhi Fei, Xueyan Niu, Pingyi Zhou, Lu Hou, Bo Bai, Lei Deng, Wei Han

Deep Exhaustive Model for Nested Named Entity Recognition (D18-1)

Challenge:	Existing models rely on a quadratic computation to generate long texts . current models impose limitations on the length of text inputs .
Approach:	They propose a semantic compression method that extends the context window of large language models . the method reduces the semantic redundancy of long inputs before passing them to the LLMs .
Outcome:	The proposed method extends the context window of large language models across tasks . it exhibits consistent fluency in text generation while reducing associated computational overhead.

Copied to clipboard

Mohammad Golam Sohrab, Makoto Miwa

LLM Jailbreak Detection for (Almost) Free! (2025.findings-emnlp)

Challenge:	Named entity recognition (NER) is a task of finding entities with specific semantic types such as Protein, Cell, and RNA in text.
Approach:	They propose a deep neural model for nested named entity recognition . they enumerate all possible regions or spans as potential entity mentions .
Outcome:	The proposed model outperforms state-of-the-art models on nested and flat NER . it achieves 77.1% and 78.4% respectively in terms of F-score, without external knowledge resources.

Copied to clipboard

Guorui Chen, Yifan Xia, Xiaojun Jia, Zhijiang Li, Philip Torr, Jindong Gu

Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home (2025.acl-long)

Challenge:	Existing methods for detecting jailbreak prompts entail significant computational costs .
Approach:	They propose a free jailbreak detection method which scales logits by temperature to detect jailbreak prompts .
Outcome:	The proposed method detects jailbreak prompts with no additional computational costs.

Copied to clipboard

Viktor Moskvoretskii, Maria Marina, Mikhail Salnikov, Nikolay Ivanov, Sergey Pletenev, Daria Galimzianova, Nikita Krayko, Vasily Konovalov, Irina Nikishina, Alexander Panchenko

Question Answering

Challenge:	Recent adaptive retrieval methods integrate LLMs’ intrinsic knowledge with external information appealing to LLM self-knowledge, but they often neglect efficiency evaluations and comparisons with uncertainty estimation techniques.
Approach:	They propose to integrate LLMs’ intrinsic knowledge with external information appealing to LLM self-knowledge but neglect efficiency evaluations and comparisons with uncertainty estimation techniques.
Outcome:	The proposed methods outperform complex pipelines in terms of efficiency and self-knowledge while maintaining comparable QA performance.

SparseFlow: Accelerating Transformers by Sparsifying Information Flows (2024.acl-long)

Copied to clipboard

Yeachan Kim, SangKeun Lee

Firewall Routing: Blocking Leads to Better Hybrid Inference for LLMs (2025.emnlp-main)

Challenge:	SparseFlow is an efficient method to sparsify the dense information flows within transformers.
Approach:	They propose a method to sparsify the dense pathways of token representations across all transformer blocks by parameterizing them to be sparse.
Outcome:	The proposed method reduces computational costs by half on average without compromising task accuracy.

Copied to clipboard

Runyu Peng, Yunhua Zhou, Kai Lv, Yang Gao, Qipeng Guo, Xipeng Qiu

Challenge:	Large language models have significantly enhanced performance across various NLP tasks . high computational costs and latency associated with deploying such models pose bottlenecks .
Approach:	They propose a dynamic hybrid inference framework that efficiently selects between a strong and a weak LLM based on the complexity of the query.
Outcome:	The proposed method outperforms existing routing strategies by up to 5.29% in APGR . large models often introduce higher latency, making them less suitable for real-time or resource-constrained applications.

CoRT: Complementary Rankings from Transformers (2021.naacl-main)

Copied to clipboard

Marco Wrzalik, Dirk Krechel

Invisible Entropy: Towards Safe and Efficient Low-Entropy LLM Watermarking (2025.emnlp-main)

Challenge:	Recent approaches to information retrieval mitigate computational costs by using a multi-stage ranking pipeline.
Approach:	They propose a ranking model that leverages contextual representations from pre-trained language models to complement term-based ranking functions while causing no significant delay at query time.
Outcome:	The proposed model significantly increases candidate recall by complementing BM25 with missing candidates while causing no significant delay at query time.

Copied to clipboard

Tianle Gu, Zongqi Wang, Kexin Huang, Yuanqi Yao, Xiangliang Zhang, Yujiu Yang, Xiuying Chen

RMoA: Optimizing Mixture-of-Agents through Diversity Maximization and Residual Compensation (2025.findings-acl)

Challenge:	Existing methods to watermark low-entropy content are expensive and risky . IE reduces parameter size by 99% while achieving performance on par with state-of-the-art methods .
Approach:	They propose a logit-based watermarking paradigm that uses entropy-based features to predict whether the next token is high or low.
Outcome:	The proposed method reduces parameter size by 99% while achieving performance on par with state-of-the-art methods.

Copied to clipboard

Zhentao Xie, Chengcheng Han, Jinxin Shi, Wenjun Cui, Xin Zhao, Xingjiao Wu, Jiabao Zhao

On the Cost-Effectiveness of Stacking of Neural and Non-Neural Methods for Text Classification: Scenarios and Performance Prediction (2021.findings-acl)

Challenge:	Multi-agent systems based on large language models are limited by high computational overhead, information loss, and robustness.
Approach:	They propose a Residual Mixture-of-Agents (RMoA) that integrates residual connections to optimize efficiency and reliability.
Outcome:	The proposed model achieves state-of-the-art performance on benchmarks of alignment, mathematical reasoning, code generation, and multitasking understanding, while significantly reducing computational overhead.

Copied to clipboard

Christian Gomes, Marcos Goncalves, Leonardo Rocha, Sergio Canuto

How Robust Are Router-LLMs? Analysis of the Fragility of LLM Routing Capabilities (2026.eacl-long)

Challenge:	Neural network algorithms excel on Automatic Text Classification tasks, but they are expensive and require high computational costs.
Approach:	They propose to exploit the cost-effectiveness of stacking of automatic text classification classifiers to improve their effectiveness.
Outcome:	The proposed method can predict the best ensemble in each scenario using only fraction of available training data.

Copied to clipboard

Aly M. Kassem, Bernhard Schölkopf, Zhijing Jin

DICE: Structured Reasoning in LLMs through SLM-Guided Chain-of-Thought Correction (2025.emnlp-main)

Challenge:	Large language model (LLM) routing has emerged as a promising solution to balancing computational costs and performance.
Approach:	They propose a framework that categorizes router performance across a broad spectrum of query types . large language models have revolutionized natural language processing .
Outcome:	The proposed framework categorizes router performance across a broad spectrum of query types . it integrates privacy and safety assessments to reveal hidden risks .

Copied to clipboard

Yiqi Li, Yusheng Liao, Zhe Chen, Yanfeng Wang, Yu Wang

BILLY: Steering Large Language Models via Merging Persona Vectors for Creative Generation (2026.eacl-long)

Challenge:	Large language models (LLMs) often prioritize reasoning over adherence to detailed instructions due to high computational costs and limited parameter access.
Approach:	They propose a lightweight framework that guides small language models to refine LLMs’ outputs through chain-of-thought correction.
Outcome:	The proposed framework improves the average format accuracy and content correctness of LLM outputs by 35.4% and 29.4%, respectively, achieving state-of-the-art (SOTA) performance over other competitive baselines.

Copied to clipboard

Tsung-Min Pai, Jui-I Wang, Li-Chun Lu, Shao-Hua Sun, Hung-yi Lee, Kai-Wei Chang

MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text (2022.emnlp-main)

Challenge:	Multi-LLM systems enhance creativity of large language models by simulating human collective intelligence but suffer from significant drawbacks, such as high computational costs and inference latency.
Approach:	They propose a training-free framework that captures the benefits of multi-LLM collaboration by extracting and blending multiple distinct persona vectors directly in the model’s activation space.
Outcome:	The proposed framework surpasses model prompting and traditional multi-LLM approaches while significantly reducing inference time and computational costs.

Copied to clipboard

Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, William Cohen

EmByte: Decomposition and Compression Learning for Small yet Private NLP (2025.findings-emnlp)

Challenge:	Pre-trained language models store a massive amount of world knowledge implicitly in their parameters, but large models often fail to encode information about rare entities and events.
Approach:	They propose a retrieval-augmented model which accesses an external non-parametric memory to augment language generation.
Outcome:	The proposed model outperforms existing models by 10-20% absolute on two datasets and under distractor and full-wiki settings.

Copied to clipboard

Shenglan Li, Jia Xu, Mengjiao Zhang

Adversarial Learning for Zero-Shot Stance Detection on Social Media (2021.naacl-main)

Challenge:	EMBYTE is a byte-level tokenization model that reduces embedding parameters by up to 94% . it is also resilient to privacy threats such as gradient inversion attacks .
Approach:	EMBYTE is a byte-level tokenization model that decomposes subwords into fine-grained byte embeddings and then compresses them via neural projection.
Outcome:	EMBYTE achieves substantial embedding compression while preserving accuracy and enhancing privacy.

Copied to clipboard

Emily Allaway, Malavika Srikanth, Kathleen McKeown

GUARD: Glocal Uncertainty-Aware Robust Decoding for Effective and Efficient Open-Ended Text Generation (2025.findings-emnlp)

Challenge:	a new model for zero-shot stance detection on Twitter uses adversarial learning to generalize across topics . previous work on zero- shot stance detector on English social media focuses on cross-target stances .
Approach:	They propose a model that uses adversarial learning to generalize across topics on Twitter . their model achieves state-of-the-art performance on unseen test topics .
Outcome:	The proposed model achieves state-of-the-art performance on unseen topics with minimal computational costs.

Copied to clipboard

Yuanhao Ding, Esteban Garces Arias, Meimingwei Li, Julian Rodemann, Matthias Aßenmacher, Danlu Chen, Gaojuan Fan, Christian Heumann, Chongsheng Zhang

Efficiently Editing Mixture-of-Experts Models with Compressed Experts (2025.findings-emnlp)

Challenge:	GUARD is a self-adaptive decoding method that balances coherence with diversity in open-ended text generation.
Approach:	They propose a self-adaptive decoding method that balances coherence and diversity . they combine global entropy estimates with local entropic deviations to integrate uncertainty .
Outcome:	GUARD achieves a good balance between diversity and coherence while exhibiting significant improvements in generation speed.

Copied to clipboard

Yifei He, Yang Liu, Chen Liang, Hany Hassan Awadalla

AdaptMerge: Inference Time Adaptive Visual and Language-Guided Token Merging for Efficient Large Multimodal Models (2025.findings-emnlp)

Challenge:	Mixture-of-Experts models allow for efficient scaling of large language models . fewer experts reduce computational costs, while more experts improve performance .
Approach:	They propose to activate only a subset of experts during training and inference . they propose compressed experts that preserve the most important experts .
Outcome:	The proposed approach preserves the most important experts while replacing other auxiliary activated experts with compressed experts.

Copied to clipboard

Zahidul Islam, Mrigank Rochan

Uni-Encoder: A Fast and Accurate Response Selection Paradigm for Generation-Based Dialogue Systems (2023.findings-acl)

Challenge:	Existing token reduction methods ignore image complexity and vision-language interactions, ignoring image complexity.
Approach:	They propose a training-free, inference-time token merging strategy that adaptively reduces visual tokens by leveraging feature diversity and language-guided relevance.
Outcome:	The proposed approach outperforms state-of-the-art token reduction methods on Google’s Gemma 3 models while achieving reduced computational costs and improved performance.

Copied to clipboard

Chiyu Song, Hongliang He, Haofei Yu, Pengfei Fang, Leyang Cui, Zhenzhong Lan

Dialogue Systems & Conversational Agents

Challenge:	Existing approaches to decode a given context-candidate pair are expensive and time-consuming.
Approach:	They propose a new paradigm that keeps full attention over each pair while only encoding the context once.
Outcome:	The proposed paradigm achieves new state-of-the-art results on four benchmark datasets with high computational efficiency.

Enrich-on-Graph: Query-Graph Alignment for Complex Reasoning with LLM Enriching (2025.emnlp-main)

Copied to clipboard

Songze Li, Zhiqiang Liu, Zhengke Gui, Huajun Chen, Wen Zhang

Spotlighter: Revisiting Prompt Tuning from a Representative Mining View (2025.findings-emnlp)

Challenge:	Existing methods employ resource-intensive, non-scalable workflows reasoning on vanilla KGs, but overlook this gap.
Approach:	They propose a flexible framework that leverages LLMs’ prior knowledge to enrich KGs and bridge the semantic gap between queries and graphs.
Outcome:	The proposed framework bridges the semantic gap between structured knowledge graphs and unstructured queries while ensuring low computational costs, scalability, and adaptability across different methods.

Copied to clipboard

Yutong Gao, Maoyuan Shao, Xinyang Huang, Chuang Zhu, Yu Weng, Xuan Liu, Lijuan Sun, Guoshun Nan

GuiLoMo: Allocating Experts and Ranks for LoRA-MoE via Bilevel Optimization with GuidedSelection Vectors (2025.findings-emnlp)

Challenge:	Spotlighter is a lightweight token-selection framework that enhances accuracy and efficiency in prompt tuning.
Approach:	They propose a token-selection framework that enhances accuracy and efficiency in prompt tuning by preserving only the top-scoring tokens for downstream prediction.
Outcome:	The proposed framework outperforms CLIP by up to 11.19% in harmonic mean accuracy and achieves 0.8K additional FPS, with only 21 extra parameters.

Copied to clipboard

Xinrong Chen, Hengyuan Zhang, Yingmin Qiu, Xiao Liang, Ziyue Li, Guanyu Wang, Weiping Li, Tong Mo, Hayden Kwok-Hay So, Ngai Wong

Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language Models (2025.findings-acl)

Challenge:	Low-Rank Adaptation (LoRA) methods are efficient for a large language model with reduced computational costs.
Approach:	They propose a layer-wise expert numbers and ranks allocation strategy with GuidedSelection Vectors.
Outcome:	The proposed method achieves superior or comparable performance to all baselines on three backbone models.

Copied to clipboard

Xi Li, Ruofan Mao, Yusen Zhang, Renze Lou, Chen Wu, Jiaqi Wang

Decoding in Latent Spaces for Efficient Inference in LLM-based Recommendation (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) have demonstrated impressive capabilities across various domains, but are vulnerable to backdoor attacks.
Approach:	They propose a chain-of-scrutiny approach which leverages LLMs’ unique reasoning abilities to mitigate backdoor attacks.
Outcome:	The proposed model is well-suited for the popular API-only LLM deployments, enabling detection at minimal cost and with little data.

Copied to clipboard

Chengbing Wang, Yang Zhang, Zhicheng Wang, Tianhao Shi, Keqin Bao, Fuli Feng, Tat-Seng Chua

UBench: Benchmarking Uncertainty in Large Language Models with Multiple Choice Questions (2025.findings-acl)

Challenge:	Light Latent-space Decoding (L2D) is an efficient and efficient latent- space decoding method.
Approach:	They propose to bypass language-space decoding by matching candidate items with LLM's internal thought representations in the latent space.
Outcome:	The proposed method is 10x faster than language-space decoding while maintaining or enhancing performance.

Copied to clipboard

Xunzhi Wang, Zhuowei Zhang, Gaonan Chen, Qiongyu Li, Bitong Luo, Zhixin Han, Haotian Wang, Zhiyu Li, Hang Gao, Mengting Hu

Improving Multi-Agent Debate with Sparse Communication Topology (2024.findings-emnlp)

Challenge:	Existing methods for benchmarking the uncertainty of large language models face challenges . existing methods require internal model access, additional training, or high computational costs .
Approach:	They propose a new benchmark for evaluating the uncertainty of large language models based on confidence intervals . UBench encompasses 11,978 multiple choice questions spanning knowledge, language, understanding, and reasoning capabilities.
Outcome:	The proposed method outperforms existing methods for benchmarking the uncertainty of large language models.

Copied to clipboard

Yunxuan Li, Yibing Du, Jiageng Zhang, Le Hou, Peter Grabowski, Yeqing Li, Eugene Ie

FiDeLiS: Faithful Reasoning in Large Language Models for Knowledge Graph Question Answering (2025.findings-acl)

Challenge:	Existing approaches to multi-agent debates use a brute force algorithm, resulting in a computationally intensive process.
Approach:	They propose to extend the multi-agent debate framework to multi-modal reasoning and alignment labeling tasks, showcasing its broad applicability and effectiveness.
Outcome:	The proposed framework can achieve comparable or superior performance while significantly reducing computational costs.

Copied to clipboard

Yuan Sui, Yufei He, Nian Liu, Xiaoxin He, Kun Wang, Bryan Hooi

Improving Preference Extraction In LLMs By Identifying Latent Knowledge Through Classifying Probes (2025.acl-long)

Challenge:	Existing retrieval-based or agent-based methods are prone to generating erroneous or hallucinated outputs.
Approach:	They propose a framework to leverage knowledge graphs as external knowledge sources to improve the factuality of LLM responses by anchoring answers to verifiable reasoning steps retrieved from KGs.
Outcome:	The proposed framework improves factuality and interpretability across benchmarks and reduces computational costs.

Copied to clipboard

Sharan Maiya, Yinhong Liu, Ramit Debnath, Anna Korhonen

Contrastive Learning for Task-Independent SpeechLLM-Pretraining (2025.findings-acl)

Challenge:	Large Language Models (LLMs) are often used as automated judges to evaluate text, but their effectiveness can be hindered by unintentional biases.
Approach:	They propose to use linear classifying probes to access LLMs’ latent knowledge and extract more accurate preferences by leveraging differences between contrasting pairs of prompts.
Outcome:	The proposed method outperforms generation-based judgements while maintaining similar computational costs.

Copied to clipboard

Maike Züfle, Jan Niehues

Frustratingly Easy Model Ensemble for Abstractive Summarization (D18-1)

Challenge:	Large language models excel in speech processing tasks but their reliance on written text limits their application in real-world scenarios.
Approach:	They propose a task-independent speech pretraining stage and task-specific fine-tuning stage to adapt LLMs to speech processing tasks.
Outcome:	The proposed model outperforms models specialized on speech translation and question answering while being trained on 10% of the task-specific data.

Copied to clipboard

Hayato Kobayashi

Achieving binary weight and activation for LLMs using Post-Training Quantization (2025.findings-acl)

Challenge:	Existing studies on compressing or distilling ensemble models have shown that they increase computational costs and reduce performance.
Approach:	They propose an unsupervised method that combines multiple models by selecting a majority-like output in post-processing.
Outcome:	The proposed method performs better than the current ensemble methods on a news-headline-generation task.

Copied to clipboard

Siqing Song, Chuang Wang, Rui-Qi Wang, Yi Yang, Xu-Yao Zhang

SPECS: Specificity-Enhanced CLIP-Score for Long Image Caption Evaluation (2025.emnlp-main)

Challenge:	Existing methods for quantizing large language models suffer from performance degradation when weights are quantized to 1 bit.
Approach:	They propose a post-training quantization framework with W(1+1)A(14) configuration . they propose utilizing Hessian-aware fine-grained grouping along with an EM-based quantization scheme .
Outcome:	The proposed method surpasses state-of-the-art (SOTA) LLM quantization baselines on W2A4 across multiple tasks.

Copied to clipboard

Xiaofu Chen, Israfel Salazar, Yova Kementchedjhieva

EffiQA: Efficient Question-Answering with Strategic Multi-Model Collaboration on Knowledge Graphs (2025.coling-main)

Challenge:	N-gram-based evaluation metrics are unreliable due to low correlation to human judgments.
Approach:	They propose a metric that rewards correct details and penalizes incorrect ones.
Outcome:	The proposed metric matches the performance of open-source LLM-based metrics in correlation to human judgments while being far more efficient.

Copied to clipboard

Zixuan Dong, Baoyun Peng, Yufei Wang, Jia Fu, Xiaodong Wang, Xin Zhou, Yongxue Shan, Kangchen Zhu, Weiguo Chen

SATER: A Self-Aware and Token-Efficient Approach to Routing and Cascading (2025.emnlp-main)

Challenge:	Existing approaches that integrate LLMs and KGs either underutilize the reasoning abilities of LLM or suffer from prohibitive computational costs due to tight coupling.
Approach:	They propose a framework that can strike a balance between performance and efficiency via an iterative paradigm.
Outcome:	The proposed framework can strike a balance between performance and efficiency via an iterative paradigm.

Copied to clipboard

Yuanzhe Shen, Yide Liu, Zisu Huang, Ruicheng Yin, Xiaoqing Zheng, Xuanjing Huang

Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling (2025.naacl-long)

Challenge:	Large language models (LLMs) demonstrate remarkable performance across diverse tasks, yet their effectiveness often depends on costly commercial APIs or cloud services.
Approach:	They propose a dual-mode compatible approach that fine-tunes models through shortest-response preference optimization and a confidence-aware rejection mechanism.
Outcome:	The proposed approach reduces redundant outputs and response times while reducing computational costs by over 50% and cascade latency by over 80%.

Copied to clipboard

Yiwen Ding, Zhiheng Xi, Wei He, Lizhuoyuan Lizhuoyuan, Yitao Zhai, Shi Xiaowei, Xunliang Cai, Tao Gui, Qi Zhang, Xuanjing Huang

CIFLEX: Contextual Instruction Flow for Sub-task Execution in Multi-Turn Interactions with a Single On-Device LLM (2025.emnlp-main)

Challenge:	Large language models (LLMs) generate solutions themselves and iteratively train on filtered, high-quality rationales, but performance reaches a ceiling after a few iterations.
Approach:	They propose a strategy to improve the efficiency of sampling heavy-tailed data by using Socratic-style guidance signals to help LLMs reasoning with complex queries.
Outcome:	The proposed approach is effective on difficult queries and on held-out tasks, while requiring human supervision.

Copied to clipboard

Juntae Lee, Jihwan Bang, Seunghan Yang, Simyung Chang

Fast Forwarding Low-Rank Training (2024.emnlp-main)

Challenge:	Experiments show that CIFLEX significantly reduces computational costs without degrading task performance.
Approach:	They propose a new execution system for efficient sub-task handling with a single large language model.
Outcome:	Experiments show that CIFLEX significantly reduces computational costs without degrading task performance.

Copied to clipboard

Adir Rahamim, Naomi Saphra, Sara Kangaslahti, Yonatan Belinkov

Ensemble Transformer for Efficient and Accurate Ranking Tasks: an Application to Question Answering Systems (2022.findings-emnlp)

Challenge:	Modern optimizers provide a spectacular array of tweaks to stabilize training trajectories and accelerate Stochastic Gradient Descent (SGD).
Approach:	They propose a fast-forward approach to accelerate large segments of SGD training . they alternate between Adam SGD for burn-in and accelerating by line search .
Outcome:	The proposed approach speeds up training without compromising model performance.

Copied to clipboard

Yoshitomo Matsubara, Luca Soldaini, Eric Lind, Alessandro Moschitti

IG-Pruning: Input-Guided Block Pruning for Large Language Models (2025.emnlp-main)

Challenge:	Large transformer models are expensive and slow to use in many applications.
Approach:	They propose an efficient neural network to distill large transformers into a single smaller model.
Outcome:	The proposed model outperforms existing models on English datasets . it outperformed existing models with 2.7 more parameters and 2.5 slower .

Copied to clipboard

Kangyu Qiao, Shaolei Zhang, Yang Feng

Knowledge-Augmented Multimodal Clinical Rationale Generation for Disease Diagnosis with Small Language Models (2025.acl-long)

Challenge:	Existing methods for efficient inference rely on fixed block masks, which can lead to suboptimal performance.
Approach:	They propose an input-aware block-wise pruning method that dynamically selects layer masks at inference time.
Outcome:	The proposed method outperforms state-of-the-art static depth pruning methods . it is particularly suitable for resource-constrained deployment scenarios .

Copied to clipboard

Shuai Niu, Jing Ma, Hongzhan Lin, Liang Bai, Zhihua Wang, Richard Yi Da Xu, Yunya Song, Xian Yang

HIGGS: Pushing the Limits of Large Language Model Quantization via the Linearity Theorem (2025.naacl-long)

Challenge:	Existing models struggle to balance predictive accuracy with human-understandable rationales.
Approach:	They propose to enhance LLMs by leveraging rationale distillation and domain knowledge injection for trustworthy multimodal rationale generation.
Outcome:	Experiments on real-world medical datasets show that ClinRaGen achieves state-of-the-art performance in disease diagnosis and rationale generation.

Copied to clipboard

Vladimir Malinovskii, Andrei Panferov, Ivan Ilin, Han Guo, Peter Richtárik, Dan Alistarh

Context Attribution with Multi-Armed Bandit Optimization (2026.findings-acl)

Challenge:	Existing methods for quantizing large language models focus on breaking down the problem into layer-wise sub-problems and minimizing per-layer error, but this approach lacks theoretical justification and the metrics employed may be sub-optimal.
Approach:	They propose a "linearity theorem" establishing a direct relationship between the layer-wise reconstruction error and the model perplexity increase due to quantization.
Outcome:	The proposed method outperforms previous data-free methods and improves accuracy-compression trade-offs on Llama-family models.

Copied to clipboard

Deng Pan, Keerthiram Murugesan, Ting Hua, Nuno Moniz, Nitesh V. Chawla

A Strategic Coordination Framework of Small LMs Matches Large LMs in Data Synthesis (2025.acl-long)

Challenge:	Existing approaches to augmenting attribution with retrieval-augmented generation (RAG) focus on training models to explicitly cite context segments during generation, but their reliability remains unverifiable.
Approach:	They propose a framework that formulates context attribution as a combinatorial multi-armed bandit problem by using Linear Thompson Sampling to efficiently identify the most influential context segments while minimizing the number of model queries.
Outcome:	The proposed method reduces model queries by 30% while matching or exceeding the attribution quality of existing approaches.

Copied to clipboard

Xin Gao, Qizhi Pei, Zinan Tang, Yu Li, Honglin Lin, Jiang Wu, Lijun Wu, Conghui He

Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator (2023.findings-acl)

Challenge:	Large Language Models suffer from high computational costs and environmental inefficiency . smaller LMs are more accessible and sustainable, but their individual capabilities often fall short . a collaborative framework for small LM combines specialized roles to iterative refinement and quality control .
Approach:	They propose a framework that aggregates specialized roles across small LMs to iterative refinement and quality control typically achieved by a single large LM.
Outcome:	The proposed framework aggregates specialized roles across small LMs to iterative refinement and quality control typically achieved by large LM.

Copied to clipboard

Ziwei He, Meng Yang, Minwei Feng, Jingcheng Yin, Xinbing Wang, Jingwen Leng, Zhouhan Lin

Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy (2021.emnlp-main)

Challenge:	Existing transformer models are computationally demanding and prohibitively costly for long sequences due to the quadratic complexity of its selfattention module.
Approach:	They propose a transformer-based model that inherits weights from large pretrained models by removing redundancies in hidden sequences using the ready-made Fast Fourier Transform operator.
Outcome:	The proposed model outperforms the standard BART model on the long-range modeling benchmark LRA with significant improvements in speed and space.

Copied to clipboard

Shaolei Zhang, Yang Feng

Disentangling Uncertainty in Machine Translation Evaluation (2022.emnlp-main)

Challenge:	Existing methods for simultaneous machine translation require multiple models for different latency levels, resulting in large computational costs.
Approach:	They propose a universal SiMT model with Mixture-of-Experts Wait-k Policy to achieve the best translation quality under arbitrary latency with only one model.
Outcome:	The proposed model outperforms all the strong baselines under different latency levels including the state-of-the-art adaptive policy.

Copied to clipboard

Chrysoula Zerva, Taisiya Glushkova, Ricardo Rei, André F. T. Martins

Machine Translation

Challenge:	Trainable evaluation metrics for machine translation (MT) exhibit strong correlation with human judgements, but they are often hard to interpret and might produce unreliable scores under noisy or out-of-domain data.
Approach:	They propose to use Monte Carlo dropout and deep ensembles to quantify uncertainty in machine translation and assess their ability to target different sources of aleatoric and epistemic uncertainty.
Outcome:	The proposed measures can target different sources of aleatoric and epistemic uncertainty, with a reduction in computational costs.

Evolving Knowledge Distillation with Large Language Models and Active Learning (2024.lrec-main)

Copied to clipboard

Chengyuan Liu, Fubang Zhao, Kun Kuang, Yangyang Kang, Zhuoren Jiang, Changlong Sun, Fei Wu

Automatic Mathematic In-Context Example Generation for LLM Using Multi-Modal Consistency (2025.coling-main)

Challenge:	Existing studies have focused on the direct use of large language models for text generation and labeling, without fully exploring their potential to comprehend the target task and acquire valuable knowledge.
Approach:	They propose to distill the knowledge of large language models into smaller models by generating annotated data.
Outcome:	The proposed method improves the performance of small domain models while enhancing the ability of large language models.

Copied to clipboard

Jaeseong Lee, Wei Yang, Gopal Gupta, Shiyi Wei

Beyond Shared Vocabulary: Increasing Representational Word Similarities across Languages for Multilingual Machine Translation (2023.emnlp-main)

Challenge:	Existing methods for in-context learning require annotated datasets, resulting in higher computational costs and lower quality examples.
Approach:	They propose a framework that automatically generates high-quality in-context examples to enhance LLMs’ mathematical reasoning.
Outcome:	Evaluated on four math problem datasets, the proposed framework outperforms baseline methods with LLM accuracy ranging from 87.0% to 99.3%.

Copied to clipboard

Di Wu, Christof Monz

Stable Language Model Pre-training by Reducing Embedding Variability (2024.emnlp-main)

Challenge:	Using a shared vocabulary is common practice in multilingual machine translation . however, when words overlap is small, e.g., using different writing systems, knowledge transfer is inhibited .
Approach:	They propose a re-parameterized method for building word embeddings using word equivalence classes and graph networks to fuse word embeds across languages.
Outcome:	The proposed method achieves evident BLEU improvements on high- and low-resource MNMT scenarios.

Copied to clipboard

Woojin Chung, Jiwoo Hong, Na Min An, James Thorne, Se-Young Yun

Energy and Carbon Considerations of Fine-Tuning BERT (2023.findings-emnlp)

Challenge:	Stable pre-training is essential for achieving better-performing language models, but tracking pre-train stability is impractical due to high computational costs.
Approach:	They propose to use Token Embedding Variability as a proxy to estimate pre-training stability.
Outcome:	The proposed method improves stability and lowers perplexities even at deeper layer counts.

Copied to clipboard

Xiaorong Wang, Clara Na, Emma Strubell, Sorelle Friedler, Sasha Luccioni

Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalization (2025.findings-emnlp)

Challenge:	Existing work quantifying energy costs and associated carbon emissions has focused on pretraining and fine-tuning.
Approach:	They perform an empirical study to quantify the energy requirements of language model fine-tuning in the context of pretraining and inference.
Outcome:	The proposed model fine-tuning energy and carbon footprints are compared with pre-training and inference energy requirements and outline recommendations for NLP researchers and practitioners.

Copied to clipboard

Shuyang Hao, Yiwei Wang, Bryan Hooi, Jun Liu, Muhao Chen, Zi Huang, Yujun Cai

ProSE: Diffusion Priors for Speech Enhancement (2025.naacl-long)

Challenge:	HKVE selectively accepts gradient optimization results based on the distribution of attention scores across different layers, ensuring that every optimization step positively contributes to the attack.
Approach:	They propose a framework that selectively accepts gradient optimization results based on the distribution of attention scores across different layers and selectively takes them into account when calculating the attack success rate.
Outcome:	The proposed framework outperforms existing methods by achieving success rates of 75.08% on MiniGPT4, 85.84% on LLaVA and 81.00% on Qwen-VL.

Copied to clipboard

Sonal Kumar, Sreyan Ghosh, Utkarsh Tyagi, Anton Jeran Ratnarajah, Chandra Kiran Reddy Evuru, Ramani Duraiswami, Dinesh Manocha

DoRA: Enhancing Parameter-Efficient Fine-Tuning with Dynamic Rank Distribution (2024.acl-long)

Challenge:	deterministic deep learning models have been used for speech enhancement, but generative models have shown promise.
Approach:	They propose a method to apply diffusion probabilistic models to speech enhancement using priors in a latent space.
Outcome:	The proposed method achieves state-of-the-art performance on synthetic and real-world datasets while consuming less computational costs.

Copied to clipboard

Yulong Mao, Kaiyu Huang, Changhao Guan, Ganglin Bao, Fengran Mo, Jinan Xu

Towards Adapting Open-Source Large Language Models for Expert-Level Clinical Note Generation (2025.findings-acl)

Challenge:	Existing parameter-efficient fine-tuning methods such as Low-Rank Adaptation ignore the differential parameter budget requirements across weight matrices, which may lead to suboptimal fine-uning outcomes.
Approach:	They propose a parameter-efficient low-rank Adaptation method that decomposes high-rank LoRA layers into structured single-rank components and allows dynamic pruning of parameter budget .
Outcome:	The proposed method outperforms LoRA and LoRA with the same parameter budget and performance.

Copied to clipboard

Hanyin Wang, Chufan Gao, Bolun Liu, Qiping Xu, Guleid Hussein, Mohamad El Labban, Kingsley Iheasirim, Hariprasad Reddy Korsapati, Chuck Outcalt, Jimeng Sun

HINT: Hypernetwork Instruction Tuning for Efficient Zero- and Few-Shot Generalisation (2023.acl-long)

Challenge:	Proprietary Large Language Models (LLMs) have demonstrated promising capabilities in clinical text summarization tasks.
Approach:	They propose a domain- and task-specific adaptation process for an open-source LLaMA-2 model . LLama-2 can generate high-quality clinical notes from outpatient patient-doctor dialogues .
Outcome:	The proposed model can generate clinical notes comparable to those authored by physicians.

Copied to clipboard

Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew Peters

Exploring the Limitations of Mamba in COPY and CoT Reasoning (2025.emnlp-main)

Challenge:	Recent NLP models have shown the remarkable ability to generalise ‘zero-shot’ to new tasks using only natural language instructions as guidance.
Approach:	They introduce Hypernetworks for INstruction Tuning (HINT) which converts task instructions and examples into parameter-efficient modules inserted into an underlying model using a pretrained text encoder.
Outcome:	The proposed models outperform strong state-of-the-art models by over 10% when controlling for compute.

Copied to clipboard

Ruifeng Ren, Zhicong Li, Yong Liu

Improved Sparse Upcycling for Instruction Tuning (2025.coling-main)

Challenge:	Inference overhead of Transformers increases linearly with the sequence length, posing challenges for modeling long sequences.
Approach:	They analyze Mamba's expressive ability to perform COPY operations and Chain of Thought reasoning tasks using a defined sequence length.
Outcome:	The proposed model can perform COPY operations and Chain of Thought reasoning tasks with a constant size while reducing computational costs.

Copied to clipboard

Wangyi Jiang, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun

Fighting Randomness with Randomness: Mitigating Optimisation Instability of Fine-Tuning using Delayed Ensemble and Noisy Interpolation (2024.findings-emnlp)

Challenge:	Existing methods for sparse upcycling lead to performance degradation in instruction tuning scenarios.
Approach:	They propose a representation-based approach to convert dense language models into sparsely activated ones by initializing router weights from language models.
Outcome:	The proposed architecture improves model capabilities and routing consistency across multiple benchmarks.

Copied to clipboard

Branislav Pecher, Jan Cegin, Robert Belanec, Jakub Simko, Ivan Srba, Maria Bielikova

Efficient OpAmp Adaptation for Zoom Attention to Golden Contexts (2025.acl-long)

Challenge:	Despite the success of fine-tuning, it still displays model performance instability, especially with limited data.
Approach:	They propose a new mitigation strategy that leverages the strengths of ensembling, noise regularisation and model interpolation while retaining computational efficiency.
Outcome:	The proposed mitigation strategy outperforms the best performing mitigation strategy (Ensemble) while using only a fraction of its cost.

Copied to clipboard

Haoyuan Wu, Rui Ming, Haisheng Zheng, Zhuolun He, Bei Yu

Disentangling Text and Math in Word Problems: Evidence for the Bidimensional Structure of Large Language Models’ Reasoning (2025.findings-acl)

Challenge:	Large language models have shown significant promise in question-answering tasks . noisy reference documents hinder performance of LLMs, causing disproportionate attention to irrelevant content .
Approach:	They propose an adaptive large language model that allocates disproportionate attention to irrelevant documents . they use transformers to train the model and integrate it into pre-trained Transformer blocks .
Outcome:	The proposed model outperforms state-of-the-art models on noisy-context benchmarks.

Copied to clipboard

Pedro Calais, Gabriel Franco, Zilu Tang, Themistoklis Nikas, Wagner Meira Jr., Evimaria Terzi, Mark Crovella

RAP: Robustness-Aware Perturbations for Defending against Backdoor Attacks on NLP Models (2021.emnlp-main)

Challenge:	Existing studies show that LLMs struggle with text interpretation and equation solving, despite distinct proficiencies in textual and mathematical components.
Approach:	They disentangle textual interpretation and mathematical solving steps in word problems drawn from Brazil's largest college entrance exam and popular grade school-level benchmark GSM8K.
Outcome:	The proposed model outperforms LLMs in Brazil's largest college entrance exam and popular grade school-level benchmark.

Copied to clipboard

Wenkai Yang, Yankai Lin, Peng Li, Jie Zhou, Xu Sun

Variator: Accelerating Pre-trained Models with Plug-and-Play Compression Modules (2023.findings-emnlp)

Challenge:	Backdoor attacks are a serious threat to the safety of reusing deep neural networks (DNNs).
Approach:	They propose an efficient online defense mechanism based on robustness-aware perturbations to distinguish poisoned and clean samples to defend against backdoor attacks on natural language processing models.
Outcome:	The proposed method achieves better defending performance and lower computational costs than existing defense methods.

Copied to clipboard

Chaojun Xiao, Yuqi Luo, Wenbin Zhang, Pengle Zhang, Xu Han, Yankai Lin, Zhengyan Zhang, Ruobing Xie, Zhiyuan Liu, Maosong Sun, Jie Zhou

LightThinker: Thinking Step-by-Step Compression (2025.emnlp-main)

Challenge:	Large language models (LLMs) have been successful on NLP tasks but require huge parameter sizes and computational resources.
Approach:	They propose a parameter-efficient acceleration method that enhances computational efficiency through plug-and-play compression plugins.
Outcome:	The proposed method saves 53% computational costs using only 0.9% additional parameters with a performance drop of less than 2%.

Copied to clipboard

Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, Ningyu Zhang

Curating Datasets for Better Performance with Example Training Dynamics (2023.findings-acl)

Challenge:	Recent advances in Large Language Models have demonstrated their remarkable capabilities in complex reasoning tasks, but their efficiency is hindered by the substantial memory and computational costs associated with generating lengthy tokens.
Approach:	They propose a method that dynamically compresses verbose thought steps into compact representations and discards original reasoning chains.
Outcome:	The proposed method reduces peak memory usage and inference time while maintaining competitive accuracy.

Copied to clipboard

Aviad Sar-Shalom, Roy Schwartz

RoseRAG: Robust Retrieval-augmented Generation with Small-scale LLMs via Margin-aware Preference Optimization (2025.findings-acl)

Challenge:	Existing methods to improve data quality but rely on data quantity to improve performance are not effective.
Approach:	They propose a method for weighing the relative importance of examples in a dataset based on their Example Training dynamics (ETD) they propose an active learning approach for computing ETD during training rather than as a preprocessing step.
Outcome:	The proposed method can be used to improve performance in in-distribution and out-of-distortion testing.

Copied to clipboard

Tianci Liu, Haoxiang Jiang, Tianze Wang, Ran Xu, Yue Yu, Linjun Zhang, Tuo Zhao, Haoyu Wang

PECAN: LLM-Guided Dynamic Progress Control with Attention-Guided Hierarchical Weighted Graph for Long-Document QA (2025.findings-acl)

Challenge:	Large language models (LLMs) have impressive performance but face high computational costs and latency, limiting their deployment in resource-constrained settings.
Approach:	They propose a robust RAG framework for large language models via Margin-aware Preference Optimization to enhance the accuracy and reliability of SLMs.
Outcome:	The proposed framework surpasses state-of-the-art benchmarks on three open-domain question answering tasks.

Copied to clipboard

Xinyu Wang, Yanzheng Xiang, Lin Gui, Yulan He

Natural Language Interfaces

Challenge:	Long-document Question Answering (QA) challenges with large-scale text and long-distance dependencies.
Approach:	They propose a method that leverages large language models to control retrieval process . they propose 'attention-based' retrieval methods that construct hierarchical graphs .
Outcome:	The proposed method achieves LLM-level performance while maintaining computational complexity comparable to RAG methods.

Self-Knowledge Guided Retrieval Augmentation for Large Language Models (2023.findings-emnlp)

Copied to clipboard

Yile Wang, Peng Li, Maosong Sun, Yang Liu

Retrieval Enhanced Feedback via In-context Neural Error-book (2025.emnlp-main)

Challenge:	Large language models (LLMs) have shown superior performance without task-specific fine-tuning due to the computational costs.
Approach:	They propose a method which lets LLMs refer to the questions they have previously encountered and adaptively call for external resources when dealing with new questions.
Outcome:	The proposed method outperforms chain-of-thought based and fully retrieval-based methods on multiple datasets and outperformed chain- of-though, chatGPT and InstructGPT.

Copied to clipboard

Jongyeop Hyun, Bumsoo Kim

QAEval: Mixture of Evaluators for Question-Answering Task Evaluation (2025.acl-long)

Challenge:	Existing methods for learning from errors lack a structured framework for analyzing and mitigating errors, especially in Multimodal Large Language Models (MLLMs).
Approach:	They propose a teacher-student framework that systematically structures errors to deliver targeted feedback for multimodal reasoning.
Outcome:	The proposed framework improves inference efficiency, token usage, and scalability by building a query-based structure that prioritizes visual information, diagnoses failure points, and guides corrective actions.

Copied to clipboard

Tan Yue, Rui Mao, Xuzhao Shi, Shuo Zhan, Zuhao Yang, Dongyan Zhao

Natural Language Interfaces

Challenge:	Existing QA evaluation methods struggle with open-ended and unstructured responses.
Approach:	They propose a hybrid framework that combines rule-based reliability with LLM-based adaptability to overcome these challenges.
Outcome:	The proposed framework outperforms existing models like GPT-4o and Claude-3 in accuracy and cost.

How Do Hyenas Deal with Human Speech? Speech Recognition and Translation with ConfHyena (2024.lrec-main)

Copied to clipboard

Marco Gaido, Sara Papi, Matteo Negri, Luisa Bentivogli

Mixture-of-Clustered-Experts: Advancing Expert Specialization and Generalization in Instruction Tuning (2025.emnlp-main)

Challenge:	Currently, attention-based models face computational hurdles in processing long sequences due to its quadratic complexity.
Approach:	They propose a conformer whose encoder self-attentions are replaced with Hyena for speech processing . they propose 'confhyena' model that reduces training time by 27% at minimal cost .
Outcome:	The proposed model reduces training time by 27% at the cost of minimal quality degradation.

Copied to clipboard

Sugyeong Eo, Jung Jun Lee, Chanjun Park, Heuiseok Lim

Text Clustering

Challenge:	A sparse Mixture-of-Experts architecture has emerged as a highly scalable solution for instruction tuning.
Approach:	They propose a mixture-of-Clustered-Experts (MoCE) architecture that allows expert specialization . they evaluate the mechanism on a set of benchmarks and show its superiority .
Outcome:	The proposed approach outperforms existing models and benchmarks on instruction tuning scenarios with significant input heterogeneity.

Adapt Once, Thrive with Updates: Transferable Parameter-Efficient Fine-Tuning on Evolving Base Models (2025.acl-long)

Copied to clipboard

Naibin Gu, Peng Fu, Xiyu Liu, Ke Ma, Zheng Lin, Weiping Wang

3DM: Distill, Dynamic Drop, and Merge for Debiasing Multi-modal Large Language Models (2025.findings-acl)

Challenge:	Parameter-efficient fine-tuning (PEFT) is a common method for fine- tuning large language models . however, once updated, PEFT modules suffer performance degradation on newer versions .
Approach:	They propose a method that enhances the PEFT module by focusing on the task-specific pattern while reducing its dependence on certain knowledge in the base model.
Outcome:	Experiments show that PEFT modules can maintain performance on updated models without re-tuning . the proposed approach can be used in real-world applications with large model sizes .

Copied to clipboard

Zhaoxi Zhang, Sanwoo Lee, Zhixiang Wang, Yunfang Wu

GraphCheck: Breaking Long-Term Text Barriers with Extracted Knowledge Graph-Powered Fact-Checking (2025.acl-long)

Challenge:	Recent advances in Multi-modal Language Models have shown remarkable performance in multimodal tasks . however, these models often exhibit inherent biases that compromise their reliability and fairness.
Approach:	They propose a framework that integrates Distill, Dynamic Drop, and Merge to address these challenges.
Outcome:	The proposed framework outperforms existing methods in balancing debiasing and improving performance on the MMSD2.0 sarcasm detection dataset.

Copied to clipboard

Yingjian Chen, Haoran Liu, Yinhong Liu, Jinxiang Xie, Rui Yang, Han Yuan, Yanran Fu, Peng Yuan Zhou, Qingyu Chen, James Caverlee, Irene Li

Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers (2024.acl-long)

Challenge:	Existing fact-checking methods that use large language models often generate subtle factual errors.
Approach:	They propose a fact-checking framework that uses extracted knowledge graphs to enhance text representation.
Outcome:	GraphCheck outperforms existing specialized fact-checkers on seven benchmarks spanning general and medical domains . Graph Neural Networks process extracted knowledge graphs as a soft prompt, enabling efficient fact- checking in a single inference call.

Copied to clipboard

Jiawen Xie, Pengyu Cheng, Xiao Liang, Yong Dai, Nan Du

Tuning Less, Prompting More: In-Context Preference Learning Pipeline for Natural Language Transformation (2025.emnlp-main)

Challenge:	Existing transformer-based models struggle with long-sequence processing due to computational costs . a framework to enhance long-content processing of transformers is proposed .
Approach:	They propose a framework to enhance long-sequence processing of transformers by three steps . they demonstrate that the framework significantly outperforms prior long-quence processors .
Outcome:	The proposed framework outperforms baseline models on long-sequence summarization and reading comprehension tasks.

Copied to clipboard

Shuyun Yang, Yan Zhang, Zhengmao Ye, Lei Duan, Mingjie Tang

Power doesn’t reside in size: A Low Parameter Hybrid Language Model (HLM) for Sentiment Analysis in Code-mixed data (2025.emnlp-main)

Challenge:	Existing approaches to natural language transformation (NLT) tasks face significant challenges, such as the computational costs of leveraging large pre-trained models and the limited generalization ability of fine-tuned smaller models.
Approach:	They propose a framework that combines prompting with fine-tuning to enhance smaller models by integrating In-Context Examples from retrieval.
Outcome:	The proposed framework outperforms existing methods across MT and TST tasks.

Copied to clipboard

Pavan Sai Balaga, Nagasamudram Karthik, Challa Vishwanath, Raksha Sharma, Rudra Murthy, Ashish Mittal

SKRAG: A Retrieval-Augmented Generation Framework Guided by Reasoning Skeletons over Knowledge Graphs (2025.findings-emnlp)

Challenge:	Code-mixed text presents significant challenges for machine learning due to interplay of distinct grammatical structures, effectively forming a hybrid language.
Approach:	They propose a Hybrid Language Model that combines a multilingual encoder and a lightweight decoder to achieve sentiment classification performance comparable to those of fine-tuned Large Language Models.
Outcome:	The proposed model outperforms models trained individually in sentiment detection tasks.

Copied to clipboard

Xiaotong Xu, Yizhao Wang, Yunfei Liu, Shengyang Li

ProFit: Leveraging High-Value Signals in SFT via Probability-Guided Token Selection (2026.findings-acl)

Challenge:	Existing KG-based question answering frameworks face inefficient subgraph retrieval, limited reasoning capabilities, and high computational costs.
Approach:	They propose a Skeleton-guided RAG framework for knowledge graph question answering . SKRAG leverages a lightweight language model enhanced with the Finite State Machine constraint .
Outcome:	The proposed framework outperforms baselines and general-domain benchmarks on a KGQA dataset in the space science and utilization domain.

Copied to clipboard

Tao Liu, Taiqiang Wu, Runming Yang, Shaoning Sun, Junjie Wang, Yujiu Yang

Challenge:	Traditional fine-tuning ignores one-to-many nature of language, leading to overfitting . authors propose a method to fine- tune LLMs by leveraging tokens.
Approach:	They propose a method to fine-tune Large Language Models by leveraging tokens to mask low-probability tokens.
Outcome:	The proposed method outperforms baselines on general reasoning and mathematical benchmarks.

The Effectiveness of Uncased Tokeniziaion for Clinical Notes (2025.findings-acl)

Copied to clipboard

Cory Paik, Katharina Von Der Wense

Syntactic Text Processing

Challenge:	a recent study shows that uncased models exceed the performance of cased models on clinical notes . case-sensitive tokenization is often seen as useful for resolving ambiguities, but for some types of text, the observed case is merely an interpretation of spoken language .
Approach:	They show that uncased models exceed the performance of cased models on clinical notes . authors propose independent case encoding to better balance model performance on cases .
Outcome:	a new study shows that uncased models outperform cased models on clinical notes . the authors say the new model improves on case-sensitive and improperly-cased tasks .

JailMeter: An Evidence-Based Evaluation Framework for Jailbreak Attacks on Large Language Models (2026.findings-acl)

Copied to clipboard

Qingjia Huang, Jingyu Zhang, Jianguo Wu, Yakai Li, Weijuan Zhang, Yankai Rong, Junyi Yao, Shengzhi Zhang, Xiaoqi Jia

The Role of Deductive and Inductive Reasoning in Large Language Models (2025.acl-long)

Challenge:	Currently, evaluation criteria and methods used for jailbreak effectiveness are inconsistent.
Approach:	They propose a framework to measure jailbreak effectiveness using a model that filters out jailbreak noise while preserving the original malicious question.
Outcome:	The proposed framework outperforms existing evaluation methods on a challenging benchmark containing 330 human-labeled, non-rejected jailbreak instances.

Copied to clipboard

Chengkun Cai, Xu Zhao, Haoliang Liu, Zhongyu Jiang, Tianfang Zhang, Zongkai Wu, Jenq-Neng Hwang, Lei Li

BiasFilter: An Inference-Time Debiasing Framework for Large Language Models (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning tasks, yet their reliability in problem-solving remains debatable.
Approach:	They propose a framework that integrates both deductive and inductive reasoning approaches to enhance LLM reasoning by progressively adapting its reasoning pathways based on problem complexity.
Outcome:	The proposed framework achieves 70.3% accuracy on AIW, compared to 62.2% for Tree of Thought, while maintaining lower computational costs.

Copied to clipboard

Xiaoqing Cheng, Ruizhe Chen, Hongying Zan, Yuxiang Jia, Min Peng

Token-level Preference Self-Alignment Optimization for Multi-style Outline Controllable Generation (2025.findings-acl)

Challenge:	Existing methods for debiasing large language models incur high human and computational costs and are limited in their effectiveness.
Approach:	They propose a model-agnostic, inference-time debiasing framework that enforces fairness by filtering generation outputs in real time.
Outcome:	The proposed framework mitigates social bias across a range of LLMs while preserving overall generation quality.

Copied to clipboard

Zihao Li, Xuekong Xu, Ziyao Chen, Lixin Zou, Ethanhjwu Ethanhjwu, Qiang Chen, Chenliang Li

Challenge:	Existing attempts to outline generation are limited by response pair requirements and substantial computation costs.
Approach:	They propose a token-level preference self-alignment optimization for outline controllable generation that extends the Bradley-Terry model from pair-wise to list-wise comparison.
Outcome:	The proposed method outperforms existing methods by 19.28% in performance while requiring only 56.25% training time.

MoE Adapter for Large Audio Language Models: Sparsity, Disentanglement, and Gradient-Conflict-Free (2026.findings-acl)

Copied to clipboard

Yishu Lei, Shuwei He, Hu Jing, Dan Zhang, Xianlong Luo, Danxiang Zhu, Shikun Feng, Rui Liu, Jingzhou HE, Yu Sun, Hua Wu, Haifeng Wang

Style-Compress: An LLM-Based Prompt Compression Framework Considering Task-Specific Styles (2024.findings-emnlp)

Challenge:	Existing research on Large Language Models (LLMs) limited to textual input modality . acoustic information is intrinsically heterogeneous, entangling attributes such as speech, music, and environmental context.
Approach:	They propose a sparse Mixture-of-Experts architecture to decouple acoustic information by routing audio tokens to specialized experts.
Outcome:	The proposed architecture outperforms existing models on audio semantic and paralinguistic tasks while retaining shared experts for global context.

Copied to clipboard

Xiao Pu, Tianxing He, Xiaojun Wan

From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression (2024.findings-emnlp)

Challenge:	Prompt compression reduces inference time and costs while maintaining informativeness for different usage scenarios.
Approach:	They propose a framework that adapts a smaller language model to compress prompts for a larger model on a new task without additional training.
Outcome:	The proposed framework outperforms two baseline models in four tasks . iteratively generates and selects effective compressed prompts as task-specific demonstrations .

Copied to clipboard

Eunseong Choi, Sunkyung Lee, Minjin Choi, Jun Park, Jongwuk Lee

Plug-and-Play Document Modules for Pre-trained Models (2023.acl-long)

Challenge:	Large language models (LLMs) have recently exhibited performance gains owing to a wide variety of prompting techniques, including Retrieval-Augmented Generation (RAG), Chain-of-Thought (CoT), and In-Context Learning (ICL).
Approach:	They propose a prompt compression method that captures the global context without compromising semantic consistency while detouring the necessity of pseudo-labels for training the compressor.
Outcome:	Empirical results show that the proposed method retains key contexts while reducing the prompt length by 80%.

Copied to clipboard

Chaojun Xiao, Zhengyan Zhang, Xu Han, Chi-Min Chan, Yankai Lin, Zhiyuan Liu, Xiangyang Li, Zhonghua Li, Zhao Cao, Maosong Sun

TwT: Thinking without Tokens by Habitual Reasoning Distillation with Multi-Teachers’ Guidance (2025.findings-emnlp)

Challenge:	Large-scale pre-trained models have been widely adopted for document-oriented NLP tasks, such as question answering.
Approach:	They propose to decouple document encoding from downstream tasks by introducing a document plugin into the backbone of a PTM.
Outcome:	The proposed model can encode documents once and for all across different scenarios.

Copied to clipboard

Jingxian Xu, Mengyu Zhou, Weichang Liu, Hanbing Liu, Shi Han, Dongmei Zhang

Nested-Refinement Metamorphosis: Reflective Evolution for Efficient Optimization of Networking Problems (2025.findings-acl)

Challenge:	Large Language Models (LLMs) have made significant strides in problem-solving by incorporating reasoning processes, but this enhanced reasoning capability results in an increased number of output tokens during inference, leading to higher computational costs.
Approach:	They propose a method that internalizes explicit reasoning into the model’s habitual behavior through a Teacher-Guided compression strategy inspired by human cognition.
Outcome:	The proposed method reduces inference-time costs while maintaining high performance while preserving high quality and diversity of the distillation dataset.

Copied to clipboard

Shuhan Guo, Nan Yin, James Kwok, Quanming Yao

ReFL: Reflective Feedback Learning for Hallucination Detection of Large Language Models (2026.acl-long)

Challenge:	Large Language Models (LLMs) excel in network algorithm design but suffer from inefficient iterative coding and high computational costs.
Approach:	They propose a method to iteratively refine task descriptions and metamorphosis on algorithms to generate more effective solutions.
Outcome:	Experimental results show that Nested-Refinement Metamorphosis outperforms state-of-the-art approaches in performance and efficiency.

Copied to clipboard

Cunhang Fan, Jun Zhang, Xue Zhang, Shuai Zhang, Zhao Lv, Jianhua Tao, Zhengqi Wen

Merging Experts into One: Improving Computational Efficiency of Mixture of Experts (2023.emnlp-main)

Challenge:	Existing methods for detecting hallucinations depend on external knowledge sources, incurring high computational costs and limiting real-time applicability, or extract the model’s internal states, leading to poor generalization.
Approach:	They propose a hallucination detection framework that leverages corrective in-context learning to guide LLMs to recognize their own prediction errors and adjust internal representations, critically without updating model weights.
Outcome:	The proposed framework outperforms existing methods on two benchmark datasets and achieves state-of-the-art performance.

Copied to clipboard

Shwai He, Run-Ze Fan, Liang Ding, Li Shen, Tianyi Zhou, Dacheng Tao

ShieldHead: Decoding-time Safeguard for Large Language Models (2025.findings-acl)

Challenge:	Extensive experiments show that MEO significantly improves computational efficiency . compared to dense networks, sparsely activated networks only employ a few parameters for each input .
Approach:	They propose a method that merges multiple experts into one to reduce computation costs . they demonstrate that a sparse Mixture of Experts (MoE) can reduce the cost by activating a small subset of parameters for each input .
Outcome:	The proposed approach reduces the computational cost to that of a single expert by 83.3% compared to 82.6% in vanilla MoE.

Copied to clipboard

Zitao Xuan, Xiaofeng Mao, Da Chen, Xin Zhang, Yuhan Dong, Jun Zhou

Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up (2025.acl-long)

Challenge:	Recent advances in LLM-based moderation methods have demonstrated remarkable promise in identifying safety risks associated with both inputs and outputs in human-AI interactions.
Approach:	They propose to learn a classification head on the last-layer hidden states of a dialogue model and use it to detect harmful content.
Outcome:	The proposed framework is 300 faster (1ms) than previous LLM-based moderation models with 99% less parameters than LlamaGuard.

Copied to clipboard

Jiahao Yuan, Dehui Du, Hao Zhang, Zixiang Di, Usman Naseem

Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models (2025.findings-acl)

Challenge:	Existing methods to improve LLMs’ logical capabilities involve traceable or verifiable logical sequences that generate more reliable responses yet increase computational costs, or introduce rigid logic template rules, reducing flexibility.
Approach:	They propose a plug-and-play reasoning framework that enhances LLMs' logical reasoning abilities during the warm-up phase prior to batch inference.
Outcome:	The proposed framework surpasses baselines in both reasoning accuracy and efficiency.

Copied to clipboard

Yingqian Cui, Pengfei He, Jingying Zeng, Hui Liu, Xianfeng Tang, Zhenwei Dai, Yan Han, Chen Luo, Jing Huang, Zhen Li, Suhang Wang, Yue Xing, Jiliang Tang, Qi He

Drilling Down into the Discourse Structure with LLMs for Long Document Question Answering (2023.findings-emnlp)

Challenge:	Chain-of-Thought (CoT) reasoning has improved the performance of large language models (LLMs) however, the detailed reasoning process in CoT often incurs long generation times and high computational costs due to the inclusion of unnecessary steps.
Approach:	They propose a method to identify critical reasoning steps using perplexity as a measure of their importance.
Outcome:	The proposed method achieves a better balance between reasoning accuracy and efficiency of CoT.

Copied to clipboard

Inderjeet Nair, Shwetha Somasundaram, Apoorv Saxena, Koustava Goswami

DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs (2025.emnlp-main)

Challenge:	Long document question answering requires locating relevant paragraphs within a document to answer a question.
Approach:	They propose to exploit the discourse structure commonly found in documents to create a condensed representation of the document, enabling a more comprehensive understanding and analysis of relationships between different parts.
Outcome:	The proposed approach retains 99.6% of the best zero-shot approach's performance while processing only 26% of tokens used by the best approach in the information seeking evidence retrieval setup.

Copied to clipboard

Minxuan Lv, Zhenpeng Su, Leiyu Pan, Yizhe Xiong, Zijia Lin, Hui Chen, Wei Zhou, Jungong Han, Guiguang Ding, Wenwu Ou, Di Zhang, Kun Gai, Songlin Hu

Stochastic Fine-Tuning of Language Models Using Masked Gradients (2024.findings-emnlp)

Challenge:	Existing sparsification methods like pruning can lose model knowledge through parameter removal.
Approach:	They propose a novel approach that achieves sparsification by partitioning pre-trained FFN layers into computational blocks.
Outcome:	The proposed approach achieves superior performance across language modeling and downstream tasks under equivalent computational constraints.

Copied to clipboard

Mohammad Akbar-Tajari, Mohammad Taher Pilehvar

The Invisible Hand: Unveiling Provider Bias in Large Language Models for Code Generation (2025.acl-long)

Challenge:	Large Language Models (LLMs) are the dominant paradigm in Natural Language Processing but fine-tuning them for specific downstream tasks often requires updating a vast number of parameters.
Approach:	They propose a method that selectively updates a small subset of parameters in each step of the tuning process.
Outcome:	The proposed approach outperforms existing fine-tuning methods while updating merely 0.08% of the model’s parameters.

Copied to clipboard

Xiaoyu Zhang, Juan Zhai, Shiqing Ma, Qingshuang Bao, Weipeng Jiang, Qian Wang, Chao Shen, Yang Liu

FLASH: Focused Layer Attention Sink Hijacking (2026.findings-acl)

Challenge:	Large Language Models (LLMs) have emerged as the new recommendation engines, surpassing traditional methods in both capability and scope, particularly in code generation.
Approach:	They propose to use a dataset to investigate a new type of bias in Large Language Models for code generation, provider bias, to determine whether the model favors specific providers.
Outcome:	The proposed model favors services from Google and Amazon, but without explicit directives, and can modify input code to incorporate their preferred providers without user requests.

Copied to clipboard

Siyuan Deng, Yerong Li, Fanhua Shang, Hongying Liu

Causal2Vec: Improving Decoder-only LLMs as Embedding Models through a Contextual Token (2026.acl-long)

Challenge:	Large Language Models (LLMs) remain vulnerable to jailbreaking attacks despite advances in safety alignment .
Approach:	They propose a new diagnostic auditing framework that dismantles the model's internal safety anchor by precisely scaling attention scores in these vulnerable layers.
Outcome:	The proposed framework achieves a state-of-the-art Attack Success Rate of over 77% with an unprecedented efficiency of 1.53 queries on average.

Copied to clipboard

Ailiang Lin, Zhuoyun Li, Yusong Wang, Kotaro Funakoshi, Manabu Okumura

OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching (2025.acl-long)

Challenge:	Existing methods modify attention mechanism to be bidirectional, undermining LLMs’ ability to extract semantic information acquired during pre-training.
Approach:	They propose a general-purpose embedding model that pre-encodes input text into a single Contextual token and then prepends it to the LLM's input sequence.
Outcome:	The proposed model improves performance of decoder-only large language models without altering their architectures or introducing significant computational overhead.

Copied to clipboard

Nghia Huynh Nguyen Hieu, Ngoc Son Nguyen, Huynh Nguyen Dang, Thieu Vo, Truong-Son Hy, Van Nguyen

GAP: a Global Adaptive Pruning Method for Large Language Models (2025.emnlp-main)

Challenge:	Text-to-speech systems have seen significant advances in recent years, driven by improvements in deep learning and neural network architectures.
Approach:	They propose a method to explore optimal transport conditional flow matching with one-step sampling and a learned prior as the condition, effectively disregarding preceding states and reducing the number of sampling steps.
Outcome:	The proposed method achieves promising performance over existing methods in content accuracy, naturalness, prosody generation, and speaker style preservation.

Copied to clipboard

Zhihua Ban, Haotian Ma, Siheng Zhang, Shengyu Liu, Xichen Chen, Ming Yang

MMS-LLaMA: Efficient LLM-based Audio-Visual Speech Recognition with Minimal Multimodal Speech Tokens (2025.findings-acl)

Challenge:	Existing structured pruning methods employ uniform compression rates across network layers, neglecting the varying importance of different network depths.
Approach:	They propose a pruning framework that minimizes global capability loss by layer-adaptive pruning rates.
Outcome:	The proposed approach achieves comparable performance with state-of-the-art methods at high pruning rates and shows significant advantages at low pruning rates.

Copied to clipboard

Jeong Hun Yeo, Hyeongseop Rha, Se Jin Park, Yong Man Ro

DCP: Dual-Cue Pruning for Efficient Large Vision-Language Models (2025.emnlp-main)

Challenge:	Recent Large Language Model (LLM) based AVSR systems incur high computational costs due to high temporal resolution of audio-visual speech.
Approach:	They propose an efficient multimodal speech LLM framework that minimizes token length while preserving essential linguistic content.
Outcome:	The proposed approach reduces token usage by 86% while using only 3.5 tokens per second.

Copied to clipboard

Lei Jiang, Zixun Zhang, Yuting Zeng, Chunzhao Xie, Tongxuan Liu, Zhen Li, Lechao Cheng, Xiaohua Xu

Reinforcement Learning for Large Language Models via Group Preference Reward Shaping (2025.emnlp-main)

Challenge:	Existing pruning methods for large vision language models use visual tokens to prune . existing methods fail to balance efficiency and semantic alignment due to large number of visual token.
Approach:	They propose a cross-modal pruning framework that considers textual semantics and visual self-attention to combine them to achieve efficient inference acceleration.
Outcome:	The proposed pruning framework can retain only 25% of the visual tokens, with a minimal performance degradation of only 0.063% on LLaVA-1.5-13B.

Copied to clipboard

Huaisheng Zhu, Siyuan Xu, Hangfan Zhang, Teng Xiao, Zhimeng Guo, Shijie Zhou, Shuyue Hu, Vasant G. Honavar

Stratified Selective Sampling for Instruction Tuning with Dedicated Scoring Strategy (2025.findings-emnlp)

Challenge:	Existing methods for fine-tuning Large Language Models (LLMs) are expensive and sensitive to reward model quality.
Approach:	They propose a method that leverages preference-based comparisons rather than precise numerical rewards.
Outcome:	Experiments show that GPRS outperforms critic-model-free RL algorithms on RLHF and reasoning tasks.

Copied to clipboard

Paramita Mirza, Lucas Weber, Fabian Küch

Consolidation or Adaptation? PRISM: Disentangling SFT and RL Data via Gradient Concentration (2026.acl-long)

Challenge:	Recent work shows that post-training datasets can be substantially downsampled without noticeably deteriorating performance.
Approach:	They propose a method that efficiently bins data into groups and scores difficulty using specialized models.
Outcome:	The proposed method can be efficient and universally applied to post-training datasets.

Copied to clipboard

Yang Zhao, Yangou Ouyang, Xiao Ding, Hepeng Wang, Bibo Cai, Kai Xiong, Jinglong Gao, Zhouhao Sun, Li Du, Bing Qin, Ting Liu

CoViPAL: Layer-wise Contextualized Visual Token Pruning for Large Vision-Language Models (2025.findings-emnlp)

Challenge:	Existing data arbitration strategies for large language model training rely on surface-level heuristics that fail to diagnose intrinsic learning needs.
Approach:	They propose a framework that arbitrates data based on its degree of cognitive conflict with the model's existing knowledge.
Outcome:	Extensive experiments on WebShop and ALFWorld show that PRISM outperforms state-of-the-art hybrid methods while reducing computational costs by up to 3.22 .

Copied to clipboard

Zicong Tang, Ziyang Ma, Suqing Wang, Zuchao Li, Lefei Zhang, Hai Zhao, Yun Li, Qianren Wang

Visual Data in NLP

Challenge:	Existing methods to prune redundant vision tokens struggle in shallow layers due to the lack of contextual information.
Approach:	They propose a layer-wise contextualized visual token pruning method that uses a plug-and-play Pruning Module to prune redundant vision tokens.
Outcome:	The proposed method outperforms training-free pruning methods under equal token budgets and surpasses training based methods with comparable supervision.

Adaptive Spatial and Temporal Redundancy Optimization for Efficient Reasoning in Large Language Models (2026.acl-long)

Copied to clipboard

Tianle Chen, Pengyu Cheng, Qiyuan Zhu, Jiacheng Wang, Bei Liu, Hao Gu, Ruijie Shen, Xiaofeng Hou, Sirui Han, Jiacheng Liu

Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning (2025.acl-long)

Challenge:	Existing research to improve CoT efficiency falls into three categories, each with distinct limitations.
Approach:	They propose a training-free framework that addresses both dimensions of CoT reasoning by applying a progressive precision reduction strategy coupled with an entropy-based confidence mechanism for adaptive termination.
Outcome:	Empirical results show that the proposed framework achieves 11.3 efficiency gain without compromising accuracy.

Copied to clipboard

Shaobo Wang, Xiangqi Jin, Ziming Wang, Jize Wang, Jiajun Zhang, Kaixin Li, Zichen Wen, Zhong Li, Conghui He, Xuming Hu, Linfeng Zhang

TELL-TALE: Task Efficient LLMs with Task Aware Layer Elimination (2026.findings-acl)

Challenge:	Using fine-tuning on task-specific data is essential for large language models to be effective in specialized tasks.
Approach:	They propose a method that leverages few-shot in-context learning with the model to be fine-tuned.
Outcome:	The proposed method outperforms existing methods with a 3.1-point improvement and a 7.4 speedup on the Llama-3-8B-Instruct model using just 10% of the dataset.

Copied to clipboard

Omar Naim, Krish Sharma, Niyar R Barman, Nicholas Asher

MutantPrompt: Prompt Optimization via Mutation Under a Budget on Modest-sized LMs (2025.findings-acl)

Challenge:	Large Language Models typically come with a fixed architecture, but not all layers contribute equally to every downstream task.
Approach:	They propose an inference-time method that selectively removes irrelevant or detrimental layers . the method is hardware-agnostic, requires no retraining, and operates entirely at inference time .
Outcome:	The proposed method matches or surpasses baseline performance while reducing computational costs.

Copied to clipboard

Arijit Nag, Animesh Mukherjee, Niloy Ganguly, Soumen Chakrabarti

CONSENSAGENT: Towards Efficient and Effective Consensus in Multi-Agent LLM Interactions Through Sycophancy Mitigation (2025.findings-acl)

Challenge:	Large Language Models (LLMs) have revolutionized the way we learn and process information, but identifying optimal prompts remains a challenge for low-resource languages.
Approach:	They propose a framework that leverages multi-armed bandit algorithms to efficiently identify optimal prompts tailored to low-resource languages.
Outcome:	The proposed framework is able to find optimal prompts for low-resource languages and significantly improves performance across multiple low-level tasks.

Copied to clipboard

Priya Pitre, Naren Ramakrishnan, Xuan Wang

LLMs as ASP Programmers: Self-Correction Enables Task-Agnostic Nonmonotonic Reasoning (2026.findings-acl)

Challenge:	Recent advances in multi-agent large language model systems have shown remarkable performance in tasks such as reasoning, planning, and decision-making.
Approach:	They propose a framework that dynamically refines prompts based on agent interactions to mitigate sycophancy by requiring additional debate rounds to reach consensus.
Outcome:	The proposed framework outperforms both single-agent and multi-a agent baselines and achieves state-of-the-art results across all benchmark datasets.

Copied to clipboard

Adam Ishay, Joohyung Lee

GMFL: Efficient Global Masking for Federated LLM Fine-tuning (2026.acl-long)

Challenge:	Recent large language models struggle with high computational costs and logical inconsistencies . a framework that translates natural language into Answer Set Programming (ASP) is developed .
Approach:	They propose a framework that translates natural language into Answer Set Programming (ASP) stable model semantics allow LLMs to express default rules and exceptions, they show .
Outcome:	The proposed framework outperforms existing methods on nonmonotonic reasoning tasks without any per-task engineering and applies uniformly across reasoning tasks.

Copied to clipboard

Xin Huang, Yan Hu, Yue-Jiao Gong, Xinglin Zhang

AIRCoder: Adaptive Integration of Multi-dimensional Retrieval for Repository-level Code Completion (2026.acl-long)

Challenge:	Low-Rank Adaptation (LoRA) has emerged as a prominent solution to mitigate the communication and computation costs in federated fine-tuning of Large Language Models (LLMs).
Approach:	They propose a plug-and-play layer freezing mechanism to integrate with existing federated fine-tuning frameworks.
Outcome:	The proposed solution reduces communication overhead and lowers computational costs while preserving the performance of the underlying federated fine-tuning methods.

Copied to clipboard

Chuanqi Shi, Miao Gao, Zhiqiang Gao

Reason Only When Needed: Efficient Generative Reward Modeling via Model-Internal Uncertainty (2026.findings-acl)

Challenge:	Existing methods for retrieving code from large codebases use textual similarity or dependency existence, resulting in inconsistent performance.
Approach:	They propose a retrieval framework that integrates eight complementary metrics across three dimensions: textual similarity, dependency existence, and structural hierarchy.
Outcome:	Experiments on CrossCodeEval and RepoEval show that AIRCoder improves accuracy and performance by 10.2 over baseline methods.

Copied to clipboard

Chao Xue, Yao Wang, Mengqiao Liu, Di Liang, Xingsheng Han, Peiyang Liu, Xianjie Wu, Chenyao Lu, Lei Jiang, Yu Lu, Haibo Shi, Shuang Liang, Minlong Peng, Flora D. Salim

LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via a Hybrid Architecture (2025.findings-emnlp)

Challenge:	Existing approaches to generating reward models rely on voting-based mechanisms to evaluate CoT outputs.
Approach:	They propose an efficient generative reward modeling framework grounded in model-internal uncertainty.
Outcome:	The proposed framework reduces inference cost while improving answer accuracy.

Copied to clipboard

Xidong Wang, Dingjie Song, Shunian Chen, Junying Chen, Zhenyang Cai, Chen Zhang, Lichao Sun, Benyou Wang

KBM: Delineating Knowledge Boundary for Adaptive Retrieval in Large Language Models (2025.findings-emnlp)

Challenge:	Long-context Large Language Models (MLLMs) are critical for video understanding and image analysis.
Approach:	They propose a hybrid architecture that integrates Mamba and Transformer blocks . they introduce data construction methods that capture both temporal and spatial dependencies .
Outcome:	The proposed model achieves competitive results across various benchmarks while maintaining high throughput and low memory consumption.

Copied to clipboard

Zhen Zhang, Xinyu Wang, Yong Jiang, Zile Qiao, Zhuo Chen, Guangyu Li, Feiteng Mu, Mengting Hu, Pengjun Xie, Fei Huang

Stability Implies Redundancy: Delta Attention Selective Halting for Efficient Long-Context Prefilling (2026.acl-long)

Challenge:	Retrieval-augmented generation (RAG) is employed to tackle these challenges . a Knowledge Boundary Model (KBM) is used to express the known/unknown of a given question .
Approach:	They propose a Knowledge Boundary Model to express the known/unknown of a given question . they find that not all questions need to trigger RAG to improve performance .
Outcome:	The proposed model reduces time and computational costs by retrieving parts of unknown knowledge . the proposed model can express the known/unknown of a given question and determine whether a RAG needs to be triggered .

Copied to clipboard

Yujie Chen, Tailai Chen, Yifeng Gao, Zoe Wanying He, Yijue Xu, Shaobo Wang, Linfeng Zhang

A Comparison of Independent and Joint Fine-tuning Strategies for Retrieval-Augmented Generation (2025.findings-emnlp)

Challenge:	Existing methods to reduce sequence length rely on heuristics that break compatibility with hardware-efficient kernels like FlashAttention.
Approach:	They propose a method that selectively halts stabilized tokens by monitoring layer-wise update dynamics of the self-attention mechanism.
Outcome:	The proposed method can reduce prefill complexity while preserving model accuracy and hardware efficiency.

Copied to clipboard

Neal Gregory Lawton, Alfy Samuel, Anoop Kumar, Daben Liu

Balcony: A Lightweight Approach to Dynamic Inference of Generative Language Models (2025.emnlp-main)

Challenge:	Multiple fine-tuning strategies exist with different costs and benefits for RAG pipelines.
Approach:	They evaluate several RAG fine-tuning strategies with different costs and benefits . embedding and generator models can be fine- tuned to increase performance .
Outcome:	The proposed techniques improve quality metrics, but have different computational costs.

Copied to clipboard

Benyamin Jamialahmadi, Parsa Kavehzadeh, Mehdi Rezagholizadeh, Parsa Farinneya, Hossein Rajabzadeh, Aref Jafari, Boxing Chen, Marzieh S. Tahaei

Cut the Deadwood Out: Backdoor Purification via Guided Module Substitution (2025.findings-emnlp)

Challenge:	Existing methods for dynamic inference are limited by hardware inefficiencies or performance degradation.
Approach:	They propose a framework for depth-based dynamic inference that freezes the pre-trained model and inserts additional transformer layers at selected exit points.
Outcome:	The proposed framework outperforms state-of-the-art methods such as Flextron and Layerskip on multiple models at various scales, as well as other leading compression techniques across a variety of benchmarks.

Copied to clipboard

Yao Tong, Weijun Li, Xuanli He, Haolan Zhan, Qiongkai Xu

Learn Like Humans: Use Meta-cognitive Reflection for Efficient Self-Improvement (2026.acl-long)

Challenge:	Model NLP models are often trained on datasets from untrusted platforms, posing significant risks of data poisoning attacks.
Approach:	They propose a retraining-free method that selectively replaces modules in the victim model based on a trade-off signal between utility and backdoor.
Outcome:	The proposed method outperforms even the strongest defense baseline against challenging attacks like LWS.

Copied to clipboard

Xinmeng Hou, Bohao Qu, Wuqi Wang, Peiliang Gong, Qing Guo, Yang Liu

Redundancy, Isotropy, and Intrinsic Dimensionality of Prompt-based Text Embeddings (2025.findings-acl)

Challenge:	Existing self-improving frameworks rely on inefficient, multi-turn recursive loops that incur high computational costs.
Approach:	They propose a framework that achieves efficient self-evolution within a single recurrence cycle.
Outcome:	The proposed framework outperforms state-of-the-art self-evolving systems while significantly reducing computational overhead.

Copied to clipboard

Hayato Tsukagoshi, Ryohei Sasano

MT-OSC: Path for LLMs that Get Lost in Multi-Turn Conversation (2026.findings-acl)

Challenge:	Prompt-based text embedding models generate task-specific embeddables but have thousands of dimensions . dimensionality reductions for embedded text can result in performance degradations of only the first 25% of the dimensions resulting in a very small degradation .
Approach:	They investigate how post-hoc dimensionality reduction affects performance of various tasks . they find that embeddings for classification and clustering exhibit lower intrinsic dimensionalities .
Outcome:	The proposed model generates task-specific embeddings upon receiving tailored prompts, but has thousands of dimensions and high storage costs.

Copied to clipboard

Jyotika Singh, Fang Tu, Miguel Ballesteros, Weiyi Sun, Sandip Ghoshal, Michelle Yuan, Yassine Benajiba, Sujith Ravi, Dan Roth

Search-in-Context: Efficient Multi-Hop QA over Long Contexts via Monte Carlo Tree Search with Dynamic KV Retrieval (2025.findings-acl)

Challenge:	Large language models suffer performance degradation when user instructions and context are distributed over multiple conversational turns.
Approach:	They propose a framework that condenses chat history in the background without disrupting the user experience.
Outcome:	The proposed framework reduces token counts by up to 72% in 10-turn dialogues while remaining robust to distractors and irrelevant turns.

Copied to clipboard

Jiabei Chen, Guang Liu, Shizhu He, Kun Luo, Yao Xu, Jun Zhao, Kang Liu

Translate Smart, not Hard: Cascaded Translation Systems with Quality-Aware Deferral (2025.emnlp-main)

Challenge:	Existing approaches to multihop question answering (MHQA) over long contexts are often neglecting explicit reasoning or incurring expensive computational costs due to full-attention mechanisms over long contextuals.
Approach:	They propose a framework that integrates Monte Carlo Tree Search (MCTS) with dynamic key-value retrieval to enable iterative, context-aware reasoning.
Outcome:	The proposed framework integrates Monte Carlo Tree Search (MCTS) with dynamic key-value (KV) retrieval to enable iterative, context-aware reasoning.

Copied to clipboard

António Farinhas, Nuno M Guerreiro, Sweta Agrawal, Ricardo Rei, Andre Martins

Leveraging Text-to-Text Transformers as Classifier Chain for Few-Shot Multi-Label Classification (2025.emnlp-main)

Challenge:	Existing quality estimation metrics are used to design effective deferral rules for machine translation.
Approach:	They propose a simple yet effective approach for machine translation using existing quality estimation metrics as deferral rules.
Outcome:	The proposed approach outperforms existing models in large translation tasks while reducing computational costs.

Copied to clipboard

Quang Anh Nguyen, Nadi Tomeh, Mustapha Lebbah, Thierry Charnois, Hanane Azzag

DRPruning: Efficient Large Language Model Pruning through Distributionally Robust Optimization (2025.acl-long)

Challenge:	Multilabel text classification (MLTC) is an essential task in NLP applications.
Approach:	They propose a distillation-based T5 generalist model for zero-shot MLTC and few-shot fine-tuning.
Outcome:	The proposed model outperforms baselines of similar size on three few-shot tasks.

Copied to clipboard

Hexuan Deng, Wenxiang Jiao, Xuebo Liu, Jing Li, Min Zhang, Zhaopeng Tu

MDTeamGPT: Mitigating Context Collapse and Enabling Self-Evolution in Medical Multi-Agent Reasoning (2026.findings-acl)

Challenge:	Structured pruning reduces model size but often causes uneven degradation across domains, leading to biased performance.
Approach:	They propose a method that dynamically adjusts the data distribution during training to restore balanced performance across heterogeneous and multi-tasking data.
Outcome:	Experiments in monolingual and multilingual settings show that the proposed method surpasses similarly sized models in pruning and continued pretraining over perplexity, downstream tasks, and instruction tuning.

Copied to clipboard

Kai Chen, Xinfeng Li, Tianpei Yang, Hewei Wang, Guang Yang, Jing Huo, Yang Gao

SEE: Strategic Exploration and Exploitation for Cohesive In-Context Prompt Optimization (2025.acl-long)

Challenge:	Long, multi-round, multirole interaction trajectories lead to severe information dilution and context window overload, triggering context collapse which destabilizes reasoning.
Approach:	They propose a multi-agent framework that compresses and reorganizes multi-round consensus.
Outcome:	The proposed framework outperforms baselines across text-based and multimodal tasks while demonstrating superior diagnostic performance and stability in complex clinical scenarios.

Copied to clipboard

Wendi Cui, Jiaxin Zhang, Zhuohang Li, Hao Sun, Damien Lopez, Kamalika Das, Bradley A. Malin, Sricharan Kumar

ReCreate: Reasoning and Creating Domain Agents Driven by Experience (2026.acl-long)

Challenge:	Existing approaches separate the optimization of prompt instructions and in-context learning examples, leading to incohesive, suboptimal results.
Approach:	They propose a framework that refines both prompt instructions and in-context learning examples.
Outcome:	The proposed framework outperforms state-of-the-art prompt optimization methods on 35 benchmark tasks.

Copied to clipboard

Zhezheng Hao, Hong Wang, Jian Luo, Jianqing Zhang, Yuyan Zhou, Qiang Lin, Can Wang, Hande Dong, Jiawei Chen

Scaling Up Temporal Domain Generalization via Temporal Experts Averaging (2025.emnlp-main)

Challenge:	Large Language Model (LLM) agents are reshaping the industrial landscape, but tasks differ widely, making them labor-intensive to build.
Approach:	They propose an experience-driven framework for the automatic creation of domain agents . they leverage agent interaction histories to provide rich concrete signals on success or failure .
Outcome:	The proposed framework outperforms human-designed agents and existing methods in experiments across diverse domains.

Copied to clipboard

Aoming Liu, Kevin Miller, Venkatesh Saligrama, Kate Saenko, Boqing Gong, Ser-Nam Lim, Bryan A. Plummer

COUNTDOWN: Contextually Sparse Activation Filtering Out Unnecessary Weights in Down Projection (2025.emnlp-main)

Challenge:	Temporal Domain Generalization (TDG) aims to generalize across temporal distribution shifts, e.g., lexical change over time.
Approach:	They propose a framework that updates the entire model using weight averaging to maximize generalization potential while minimizing computational costs.
Outcome:	The proposed framework outperforms previous methods by up to 69% while being up to 60x more efficient.

Copied to clipboard

Jaewon Cheon, Pilsung Kang

Less is More: Knowledge-Aware Compression for Long Legal Judgment Prediction (2026.findings-acl)

Challenge:	Large language models have created significant computational inefficiencies due to their size and complexity.
Approach:	They propose to use a linear combination to deactivate non-essential parameters during inference to reduce computational costs.
Outcome:	The proposed methods can omit 90% of computations with performance loss as low as 5.5% ideally, while M-COUNTDOWN provides a predictor-free solution with up to 29.4% better performance preservation compared to existing methods.

Copied to clipboard

Fanghao Lou, Qiqi Wang, Guanyu Chen, Senbo Zhang, Kaiqi Zhao, Qian Liu, Huijia Li

Accelerating Training of Autoregressive Video Generation Models via Local Optimization with Representation Continuity (2026.findings-acl)

Challenge:	Recent advances leverage large language models (LLMs) for legal reasoning, but they face high computational costs and information degradation when handling long cases.
Approach:	They propose a framework that selectively retains legally relevant information while reducing redundant or less informative content, enabling efficient and accurate long-context reasoning.
Outcome:	The proposed framework outperforms existing methods on four real-world datasets spanning multiple jurisdictions and languages.

Copied to clipboard

Yucheng Zhou, Jianbing Shen

Representation Learning

Challenge:	Existing autoregressive models have shown superior performance and efficiency in image generation, but are constrained by high computational costs and prolonged training times in video generation.
Approach:	They propose a Local Optimization method which optimizes tokens within localized windows while leveraging contextual information to reduce error propagation.
Outcome:	The proposed method achieves superior performance to the baseline while halving the training cost without sacrificing quality.

TALAS: Teacher-Anchored Layer Alignment with Adaptive Sharpness-Aware Minimization for Embedding Distillation (2026.acl-long)

Copied to clipboard

Quoc Phong Dao, Hoang Son Nguyen, Pham Khanh Chi, Linh Ngo Van, Nguyen Thi Ngoc Diep, Thien Huu Nguyen, Trung Le

LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents (2025.emnlp-main)

Challenge:	Existing methods that require a student to strictly mimic the teacher’s sentence embeddings or internal features often incur prohibitive computational costs and yield suboptimal performance due to the inherent capacity gap.
Approach:	They propose a Teacher-Anchored mechanism that selectively distills final sentence embeddings only into the student’s upper layers, thereby reducing overhead while respecting capacity constraints.
Outcome:	Empirical results show that TALAS outperforms existing methods while maintaining high performance.

Copied to clipboard

Taro Yano, Yoichi Ishibashi, Masafumi Oyamada

Challenge:	Existing approaches to optimize large language models rely on manual design or focus on optimizing individual components.
Approach:	They propose a LaMDAgent framework that constructs and optimizes end-to-end post-training pipelines by exploring various model improving methods, objects, and their applied orderings based on task-based feedback.
Outcome:	The proposed framework achieves a 9.0-point gain in tool-use accuracy without degrading instruction-following, and reduces computational costs.

Squeezed Attention: Accelerating Long Context Length LLM Inference (2025.acl-long)

Copied to clipboard

Coleman Richard Charles Hooper, Sehoon Kim, Hiva Mohammadzadeh, Monishwaran Maheswaran, Sebastian Zhao, June Paik, Michael W. Mahoney, Kurt Keutzer, Amir Gholami

SeDev: Structured Semantic Exploration for LLM-Driven Code Generation (2026.acl-long)

Challenge:	Emerging Large Language Models require long input context to perform complex tasks.
Approach:	They propose an algorithm to reduce the complexity of attention with respect to the fixed context length.
Outcome:	The proposed method reduces the complexity of attention from linear to logarithmic with respect to the fixed context length.

Copied to clipboard

Ronghui Yang, Jie Liu, Jiajie Zeng, Jiexin Wang, Jiuchuan Jiang, Bo An, Yi Cai, Mengchen Zhao

Programming Languages in NLP

Challenge:	Large Language Models (LLMs) have shown remarkable capabilities in automating code generation, but they suffer from insufficient exploration of the vast solution space.
Approach:	They propose a large-scale LLM-driven code generation framework that efficiently finds high-quality solutions in only a few iterations.
Outcome:	The proposed framework outperforms baselines while maintaining reasonable time and computational costs.

Pru-CoT: Towards Efficient Reasoning Distillation via Pruning Chain-of-Thought (2026.findings-acl)

Copied to clipboard

Han Liu, Shuotian Ma, Hui Li, Xiaotong Zhang, Fenglong Ma, Hong Yu

NormXLogit: The Head-on-Top Never Lies (2025.emnlp-main)

Challenge:	Existing heuristics fail to capture global causal logic due to rigid rules and limited search spaces.
Approach:	They propose a framework that extracts the essential logical structure from reasoning chains.
Outcome:	Experiments show that Pru-CoT models generate more compact reasoning paths compared to models trained on verbose data.

Copied to clipboard

Sina Abbasi, Mohammad Reza Modarres, Mohammad Taher Pilehvar

Explainability & Interpretability in NLP

Challenge:	NormXLogit is a novel approach for assessing the significance of input tokens based on word embeddings .
Approach:	They propose a novel method for assessing the significance of input tokens based on the input and output representations associated with each token.
Outcome:	The proposed method outperforms gradient-based methods in faithfulness and offers competitive performance compared to leading architecture-specific techniques.

Polymorphic Universal Transformer (2026.acl-long)

Copied to clipboard

Yilong Chen, Zitian Gao, Yihao Xiao, Jason Klein Liu, Xinyu Yang, Yifan Luo, Haoming Luo, Zhengmao Ye, Tingwen Liu, Ran Tao, Bryan Dai

GROKE: Vision-Free Navigation Instruction Evaluation via Graph Reasoning on OpenStreetMap (2026.acl-long)

Challenge:	Compute Distribution Skew is a pathological phenomenon in ultra-deep recurrent models . it causes over-smoothing, representation rank collapse, and degraded reasoning performance.
Approach:	They propose a dynamic architecture that redefines recursive computation by decoupling parameter count from depth.
Outcome:	The proposed model significantly improves representation rank and reasoning robustness while reducing computation by 64.7%.

Copied to clipboard

Farzad Shami, Subhrasankha Dey, Nico Van de Weghe, Henrikki Tenkanen

CAP: Controllable Alignment Prompting for Unlearning in LLMs (2026.acl-long)

Challenge:	Existing evaluation metrics for navigation instructions are limited by licensing constraints and computational costs.
Approach:	They propose a vision-free training-free hierarchical LLM-based framework for evaluating navigation instructions using OpenStreetMap data.
Outcome:	The proposed framework reduces navigation error by 68.5% compared to baselines on the Map2Seq dataset.

Copied to clipboard

Zhaokun Wang, Jinyu Guo, Jingwen Pu, Hongli Pu, Meng Yang, Xunlei Chen, Jie Ou, Wenyi Li, Guangchun Luo, Wenhong Tian

Valid Necessary: Diagnosing Latent Inefficiency in Chain-of-Thought (2026.findings-acl)

Challenge:	Existing methods for modifying parameters are unsystematic and rely on empirical experience.
Approach:	They propose a controllable alignment prompting for unlearning framework that decouples unlearning into a learnable prompt optimization process via reinforcement learning.
Outcome:	The proposed framework achieves precise, controllable unlearning without updating model parameters.

Copied to clipboard

Daeyeop Lee, Hwanjo Yu

ProMCP: Profiling Token Flows and Latency Costs in Model Context Protocol–Based LLM Agents (2026.findings-acl)

Challenge:	Existing reasoning step evaluators fail to distinguish “valid but inefficient” reasoning steps from necessary reasoning.
Approach:	They propose a training-free metric that identifies low-utility steps and a post-hoc compression strategy to quantify their impact on token usage.
Outcome:	The proposed metric reduces token consumption by 31–53% while maintaining accuracy at substantially higher compression rates.

Copied to clipboard

Sumera Anjum, Weijian Zheng, Rajkumar Kettimuthu, Heng Fan, Yunhe Feng

Supplement Generation Training for Enhancing Agentic Task Performance (2026.findings-acl)

Challenge:	Large Language Models are increasingly used as agents that interact with external tools and data sources to solve tasks that require fresh knowledge, precise computation, or action in a real environment.
Approach:	They propose a framework that decomposes the MCP workflow into a six-stage communication pipeline and enables granular attribution of computational costs.
Outcome:	The proposed framework decomposes the MCP workflow into a six-stage communication pipeline, enabling granular attribution of computational costs.

Copied to clipboard

Young Min Cho, Daniele Bonadiman, Divya Bhargavi, Tamer Alkhouli, Salvatore Romeo, Dongwei Jiang, Khushbu Pahwa, Yubin Ge, Etsuko Ishii, Monica Sunkara, Yi Zhang

Challenge:	Training large foundation models for agentic tasks is impractical due to high computational costs, long iteration cycles, and rapid obsolescence as new models are released.
Approach:	They propose a method that trains a small LLM to generate supplemental text that helps the larger LLM solve the task more effectively.
Outcome:	The proposed approach decouples task-specific optimization from large foundation models . it achieves consistent and significant performance gains across diverse tasks and models - all without gradient access to the actor model.

Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning (2026.findings-acl)

Copied to clipboard

Zichuan Fu, Xian Wu, Guojing Li, Yejing Wang, Yijun Chen, Zhao Zihao, Luo Yixuan, Hanyu Yan, Yefeng Zheng, Xiangyu Zhao

Seer Self-Consistency: Advance Budget Estimation for Adaptive Test-Time Scaling (2026.findings-acl)

Challenge:	Recent advances in large language models (LLMs) have catalyzed the rise of reasoningintensive inference paradigms, where models perform explicit step-by-step reasoning before generating final answers.
Approach:	They propose a large-small LLM collaboration framework that synergizes large and small language models to achieve high-quality reasoning with significantly reduced computational cost.
Outcome:	The proposed framework outperforms the mentor LLM while preserving the benefits of the thinking paradigm of LLMs.

Copied to clipboard

Shiyu Ji, Yixuan Wang, Yijun Liu, Qingfu Zhu, Wanxiang Che

Challenge:	Large Language Models (LLMs) have demonstrated exceptional performance in reasoning tasks, particularly in mathematics.
Approach:	They propose a dynamic self-consistency framework that integrates System 1 and System 2 reasoning to improve token efficiency and latency.
Outcome:	The proposed method outperforms existing methods, achieving up to 47% reduction in token consumption and 43% reduction in inference latency without significant performance loss.

Thinking-Based Non-Thinking: Solving the Reward Hacking Problem in Training Hybrid Reasoning Models via Reinforcement Learning (2026.acl-long)

Copied to clipboard

Siyuan Gan, Jiaheng Liu, Boyan Wang, Tianpei Yang, Runqing Miao, Yuyao Zhang, Fanyu Meng, Junlan Feng, Linjian Meng, Jing Huo, Yang Gao

Dictionary Guided Sparse Logit Editing for Reliable Jailbreak Attacks (2026.findings-acl)

Challenge:	Existing work on large reasoning models (LRMs) focuses on using reinforcement learning (RL) to train hybrid reasoning models that automatically decide whether to engage in thinking or not based on the complexity of the query.
Approach:	They propose to use reinforcement learning to train hybrid reasoning models that automatically decide whether to engage in thinking or not based on the complexity of the query.
Outcome:	The proposed model reduces token usage by around 50%$ compared to DeepSeek-R1-Distill-Qwen-1.5B/7B and DeepScaleR-1.5b, while significantly improving accuracy.

Copied to clipboard

Shuaibiao Han, Ruiyang Ni, Zhiyu Yi, Changlong Li, Perley Xu, Wenjie Ruan